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1.  Introduction 


The  recently  concluded  8-year  Robotics  Collaborative  Technology  Alliance  (RCTA)  pursued 
technologies  to  support  tactical  behaviors  of  unmanned  ground  systems  as  a  way  to  illustrate 
advances  in  the  research  program  thrust  areas  of  perception,  intelligent  control,  and  Soldier- 
machine  interface.  In  April  2009,  the  U.S.  Army  Research  Laboratory  (ARL)  conducted  an 
experiment  at  Fort  Indiantown  Gap  (FTIG),  PA,  to  measure  the  capability  of  current  autonomous 
navigation  planning  algorithms  to  solve  complex  terrain  challenges  previously  not  achievable 
under  the  local  planning  approach  used  in  Demo  III,  the  forerunner  program  and  baseline  for  the 
RCTA.  The  principal  focus  was  to  evaluate  the  relative  performance  of  three  planning 
algorithms  representing  the  span  of  research  from  Demo  III  to  the  closing  months  of  the  RCTA. 

The  approaches  evaluated  were  the  (1)  Demo  III  Autonomous  Mobility  (AM)  Planner  with  local 
planning  only,  (2)  Dynamic  Replanner  (DR)  with  shared  sensed  data  and  a  continuous  update  of 
the  global  plan,  and  (3)  Field  Cost  Interface  (FCI)  Planner,  which  continuously  generated  a 
combined  cost  (local  +  global)  by  combining  the  kinematic  constraints  of  the  vehicle  and  the 
recommendations  of  the  global  planner.  The  Dynamic  and  FCI  planners  involve  two-level 
autonomous  mobility  planning  to  solve  route  challenges,  adding  to  the  local  planner  established 
in  Demo  III  the  ability  to  update  the  global  map  with  sensed  data  and  to  revise  the  route  with 
both  local  and  global  information  considered.  An  additional  High-Maneuverability  Planner 
(HMP)  was  available  to  DR  for  local  repositioning  to  make  feasible  the  execution  of  dynamically 
determined  routes,  hence  calling  for  a  change  in  direction  outside  the  kinematic  constraints  of  the 
platform. 

Prior  to  the  field  experiment,  the  experimental  conditions  were  modeled  in  robotic  interactive 
visualization  and  exploitation  technology  (RIVET),  a  high-fidelity  hardware-in-the-loop 
simulator,  and  the  experimental  design  was  executed  in  simulation.  Simulating  the  experiment 
served  to  (1)  confirm  the  feasibility  and  level  of  challenge  for  the  proposed  route,  and  (2) 
establish  a  reference  set  for  the  expected  performance  of  the  planners  evaluated.  The  design 
included  seven  route  scenarios  of  varying  difficulty  and  three  planners.  Three  replications  of  a 
full  factorial  design  (63  runs)  were  run  in  RIVET.  Each  scenario  had  an  achievable  goal  point 
(based  on  natural  terrain  alone),  but  each  route  was  physically  blocked  to  create  a  cul-de-sac 
impassable  by  the  experimental  unmanned  vehicle  (XUV)  under  the  initial  plan.  Thus,  in  order 
for  a  planner  to  successfully  navigate  the  XUV  to  the  goal,  an  alternative  route  had  to  be  found, 
first  out  of  the  cul-de-sac  and  then  to  the  goal. 

In  the  field  experimentation  at  FTIG,  at  least  two  replications  of  each  scenario  were  attempted 
for  each  planner.  The  AM  planner  reached  the  goal  on  33%  of  the  runs,  FCI  achieved  48%  of 
the  goal  points,  and  DR  achieved  64%  of  the  goal  points.  A  distinction  in  performance  was 
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made  between  achieving  the  goal  point  and  solving  the  intermediate  cul-de-sac  challenge.  The 
AM  planner  solved  40%  of  the  cul-de-sacs,  FCI  solved  52%,  and  DR  solved  79%.  This  is  a 
major  improvement  in  maneuvering  in  complex  terrain  and  is  a  technology  enabler  for  more 
complex  tactical  behaviors  and  reduced  operator  workload. 

RIVET  was  successful  in  predicting  about  half  (48%)  of  the  field  run  performance,  but  the  lack 
of  run-to-run  variability  in  RIVET  hindered  its  usefulness  in  building  a  reference  distribution  of 
performance.  Its  use  as  a  planning  tool,  however,  was  invaluable.  The  feasibility  and  level  of 
challenge  confirmed  in  simulation  translated  to  the  field  extremely  well,  so  that  differences  in 
performance  anticipated  among  the  three  planners  were  observed  in  the  field.  Setting  an 
appropriate  level  of  challenge  for  an  experiment  had  been  elusive  in  earlier  studies  without 
RIVET,  resulting  in  some  inefficient  testing. 


2.  Background 


2.1  History 

In  2000,  the  ARE  Demo  III  unmanned  ground  vehicle  (UGV)  technology  program  demonstrated 
integrated  UGV  perception  and  planning  for  autonomous  navigation  in  unstructured 
environments.  The  purpose  of  Demo  III  was  to  develop  UGV  technology  that  enabled  robust 
autonomous  mobility  in  relevant  environments.  The  focus  for  UGV  performance  measurement 
was  twofold:  (1)  to  determine  the  level  of  maturity  of  autonomous  mobility  technology  and  (2) 
to  measure  the  impact  on  Soldier  workload  when  operator  intervention  is  required  to  continue 
UGV  operation.  In  2002  and  2003,  the  Demo  III  program  conducted  a  series  of  experiments  at 
three  sites  that  ascertained  the  technology  readiness  level  (TRL)  of  autonomous  navigation  for 
UGVs.'  The  UGV  platform  was  the  XUV.  With  regard  to  mobility,  the  accomplishment  in 
Demo  III  was  the  enhanced  local  navigation  around  obstacles  as  the  XUV  followed  a 
predetermined  waypoint  route  up  to  2  km  long.  Successful  performance  in  a  relevant 
environment  earned  a  TRL-6  designation  for  the  autonomous  navigation  system  at  that  time. 
Since  2003,  technology  developments  under  the  ARE  RCTA  have  been  measured  in  terms  of 
their  contribution  to  notional  mission  profiles  envisioned  by  the  Future  Combat  Systems 
Program.'  Generally,  two  tracks  of  experimentation  have  been  followed:  (1)  tactical  behaviors, 
especially  as  to  conducting  reconnaissance  and  (2)  safe  operations  when  the  UGV  is  operating  in 
close  proximity  to  humans.* 


^Camden,  R.;  Bodt,  B.;  Schipani,  S.;  Bomstein,  J.;  Runyon,  T.;  French,  F.;  Shoemaker,  C.;  Jacoff,  A.;  Lytle,  A.  Autonomous 
Mobility  Technology  Assessment  Final  Report,  ARL-TR-3471;  U.  S.  Army  Research  Laboratory:  Aberdeen  Proving  Ground, 
MD,  2005. 
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2.2  Purpose  of  the  Experiment 

The  purpose  of  this  experiment  is  to  measure  the  advances  in  autonomous  navigation  under 
extremely  challenging  scenarios.  In  the  TRL-6  study,  navigation  was  stressed  primarily  by 
vegetation  and  natural  obstacle  challenges,  impacting  the  way  toward  the  next  waypoint  of  a 
predetermined  route.  In  this  study,  the  challenge  was  more  austere.  Planning  algorithms  were 
used  to  guide  the  XUV  between  waypoints  and  around  natural  obstacles  and,  to  be  more  flexible 
in  completing  a  mission,  to  untether  the  robot  from  a  predetermined  waypoint  route  to  move 
tactically  to  a  goal  point.  This  autonomous  tactical  movement  required  the  planner  to  resolve 
incomplete  or  erroneous  terrain  information  or  unexpected  terrain  situations,  making  the  original 
planned  path  impossible  to  traverse. 

To  accomplish  tactical  movement,  the  methods  for  resolving  local  challenges  had  to  evolve. 
Since  the  time  of  Demo  III  and  the  TRL-6  experiment,  when  faced  with  a  situation  where  no 
traversable  local  path  could  be  found  by  the  XUV,  the  method  to  finding  an  alternate  path  has 
been  to  successively  back  up  (5,  10,  and  15  m)  and  search  for  an  acceptable  local  path  to  the  next 
waypoint  through  the  terrain  ahead  using  a  sensor-based  near- field  (~40  m)  representation  of  the 
environment.  This  approach  emphasizes  the  view  in  front  of  the  XUV  and  assumes  that  the 
planned  path  is  achievable  if  a  small  tolerance  is  given  to  the  XUV  to  deviate  from  that  planned 
path  around  any  local  obstacle.  This  approach  (local  planning  only)  had  a  limited  capability 
because  it  required  frequent  operator  assistance  in  order  to  continue  to  the  goal  point.  Locally 
sensed  data  remained  at  the  local  level.  Higher-level  planning  relied  upon  static,  a  priori  data 
from  an  external  source  and  resulted  in  a  predetermined  waypoint  route. 

Research  on  two-level  (local  and  global)  planning  was  a  major  initiative  in  the  RCTA.  Sensed 
terrain  data  would  be  shared  from  the  local  to  the  global  level,  and  routes  would  potentially  be 
adjusted  in  light  of  the  new  information.  It  was  believed  that  new  methods  for  finding 
alternative  paths  in  consideration  of  a  goal  point  would  result.  The  autonomy  of  the  vehicle 
would  be  enhanced,  and  the  operator’s  workload  would  be  reduced. 

No  operator  interventions  (teleop,  manual  backup,  etc.)  were  permitted  in  this  experiment  to 
keep  the  focus  on  the  technologies  designed  to  improve  autonomous  mobility. 

2.3  Three  Autonomous  Navigation  Approaches 

In  recent  years,  researchers  from  the  RCTA  have  developed  several  new  approaches  to 
combining  local  and  global  planning  into  a  hierarchical  approach  to  autonomous  navigation.^’^ 
These  approaches  have  the  potential  to  enable  the  XUV  to  overcome  complex  terrain  challenges 
and  significantly  reduce  the  need  for  operator  assistance  in  successfully  completing  a  tactical 

2 

Childers,  M.;  Bodt,  B.;  Hill,  S.;  Camden,  R.  The  Impaet  of  Multi-Level  Path  Planning  on  Unmanned  Ground  Vehiele 
Taetieal  Behavior;  26th  Army  Seienee  Conferenee,  Orlando,  FL,  Deeember  2008. 
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Childers,  M.;  Bodt,  B.;  Hill,  S.;  Camden,  R.;  Dean,  R.;  Dodson,  W.;  Sutton,  L.;  Sapronov,  L.  Unmanned  Ground  Vehicle 
Tactical  Behaviors  Technology  Assessment,  ARL-TR-4698:  U.S.  Army  Researeh  Laboratory:  Aberdeen  Proving  Ground,  MD, 
2009. 
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movement.  The  ultimate  goal  for  the  development  of  such  planning  algorithms  is  to  achieve 
adaptive  control  in  dynamic,  unfamiliar  situations.  The  experiment  focused  on  measuring  the 
performance  differences  among  three  planning  approaches  and  assessing  the  benefit  they  provide 
to  the  UGV  when  faced  with  challenging  situations. 

2.3.1  Autonomous  Mobility 

In  this  mode,  the  XUV  receives  an  initial  global  route  generated  by  the  Command  and  Control 
global  planner  from  a  priori  data.  The  XUV  then  uses  the  perception  system  and  the  local  AM 
planner  to  follow  this  route  and  avoid  obstacles  along  the  way.  The  local  planner  considers  the 
kinematic  constraints  of  the  XUV  but  has  no  knowledge  of  conditions  beyond  the  sensor  range 
(~40  m).  In  this  mode,  there  are  no  updates  to  the  global  route,  and  the  XUV  will  most  likely 
fail  to  find  an  alternative  if  there  are  blockages  that  invalidate  the  original  route  and  that  cannot 
be  avoided  with  local  maneuvers. 

Figure  1  shows  the  initial  plan  generated  from  a  priori  data  from  the  start  point  (SP)  to  the  goal. 
The  AM  planner  would  attempt  to  follow  this  route  and  would  normally  do  so  successfully  if  the 
a  priori  data  is  reasonably  accurate. 


Figure  1 .  An  initial  route  planned  by  the  global  planner. 

2.3.2  Dynamic  Replanner 

The  DR  makes  use  of  sensed  terrain  data  passed  from  local  to  global  maps  (called  best 
information  planning).  The  global  map  is  now  continuously  improved  with  current,  locally 
sensed  terrain  information.  In  this  mode,  the  global  planner  generates  an  initial  optimal  route 
based  on  the  a  priori  feature  data.  As  the  global  map  is  updated,  a  new  global  route  is 
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continuously  generated  and  compared  to  the  current  route.  When  a  sufficient  difference  in  the 
routes  occurs,  the  new  global  route  is  sent  to  the  XUV.  The  global  plan  always  ignores  the 
kinematic  constraints  of  the  XUV,  so  if  the  route  has  an  initial  heading  that  differs  from  the 
vehicle’s  current  heading  by  more  than  45°,  the  HMP  is  called  to  reposition  the  vehicle  before 
following  the  intended  route. 

Figure  2  illustrates  a  new  global  plan  generated  by  the  DR  after  sensing  the  blockage  on  the 
original  planned  route,  and  figure  3  illustrates  a  new  global  plan  at  the  point  where  it  sensed  that 
the  first  alternative  route  was  also  blocked. 


Figure  2.  An  alternate  route  generated  by  the  DR. 


Figure  3.  Second  alternative  route  generated  by  the  DR. 
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The  HMP  is  a  search-based,  nonholonomic  planner  that  models  not  only  the  position  {x,y)  of  the 
XUV,  but  also  its  heading  and  its  limited  turning  radius.  As  such,  it  is  able  to  provide  paths  that 
are  kinematically  correct  and  that  are  not  limited  to  a  predetermined  set  of  egographs.  The 
planner  has  a  small  set  of  primitive  motions  that  are  repeated  over  the  search  space,  generating 
complex  trajectories  that  adapt  to  the  constraints  of  the  environment.  This  planner  is  able  to 
automatically  generate  maneuvers,  such  as  three-point  turns  and  “k”  turns.  However,  because  of 
the  complexity  of  its  planning  space,  in  a  few  seconds  HMP  can  only  plan  trajectories  within  a 
small  region  near  the  vehicle  (~20  m).  Figure  4  illustrates  an  HMP-planned  maneuver  when  the 
dynamic  replanner  sends  a  new  plan  that  begins  more  than  45°  from  the  current  trajectory. 


Figure  4.  FIMP-generated  maneuver. 


2.3.3  Field  Cost  Interface 

The  FCI  planner  continuously  generates  a  cost  field  at  a  radius  R  from  the  vehicle,  using  both 
prior  data  and  sensed  data.  FCI  then  generates  a  combined  cost  (local  +  global)  at  each  point  in 
the  circle  defined  by  the  radius  R,  thereby  combining  the  kinematic  constraints  of  the  vehicle  and 
the  recommendations  of  the  global  planner.  The  lowest  combined  cost  becomes  the  new  local 
plan.  Figure  5  illustrates  the  field  costs  at  the  “donut”  surrounding  the  XUV  as  it  moves  toward 
the  first  blockage  in  scenario  5. 
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Figure  5.  Field  costs  at  cul-de-sac  in  scenario  5. 


The  FCI’s  performance  depends  on  the  weight  that  global  costs  are  given  when  combined  with 
local  costs: 


Ctotal  =  C’local  +  CellScale*C’giobal ,  (1) 

where  C’locai  and  C’giobai  are  scaled  local  and  global  path  costs.  Prior  to  this  assessment,  the  value 
of  CellScale  was  tuned  using  RIVET  simulations  and  field  experiments  in  the  fall  of  2008. 

Based  on  a  limited  number  of  runs,  a  value  of  0.2  was  determined  to  perform  best  and  was  the 
value  chosen  for  this  assessment.  Unlike  with  the  DR,  the  HMP  has  not  been  integrated  with  the 
FCI  planner,  putting  FCI  at  a  disadvantage  where  the  ability  to  maneuver  in  a  tight  space  is 
required. 


3.  Robotic  Interactive  Visualization  and  Exploitation  Technology  (RIVET) 


RIVET  is  a  high-fidelity  hardware-in-the-loop  simulator  that  not  only  generates  a  rich  world 
representation  in  real  time,  but  also  simulates  sensors  and  the  interactions  of  the  vehicle  with  the 
terrain  and  other  moving  entities,  while  having  the  same  low-level  interface  as  the  actual 
hardware  in  the  robotic  vehicle. 

Developed  under  the  RCTA,  RIVET  features  a  game-based,  interactive,  multiuser  modeling  and 
simulation  platform  specifically  suited  to  developing  perception,  planning,  and  other  robotic 
technologies  faster  and  more  affordably.  RIVET ’s  feature-rich  environment  and  scalability 
allow  users  to  configure  the  system  to  include  pedestrians  as  well  as  air-,  ground-,  and  sea-based 
robotic  assets  so  that  realistic  small-team  tasks  can  be  played  out  safely  and  repetitively  in  the 
virtual  world. 
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By  leveraging  the  latest  developments  in  game  technology  and  graphics  hardware,  RIVET  is  able 
to  provide  stunning  graphics,  special  effects,  high-fidelity  physics  simulation,  terrain  rendering, 
sensor  simulation,  and  weather  effects,  such  as  fog  and  rain. 

3.1  Representing  Mobility  Features 

The  current  representation  used  in  RIVET  allows  limited  modeling  of  local  terrain  characteristics 
that  excludes  ruts,  shoulders,  gravel,  etc.  As  a  result,  subsystems  that  detect  and  use  these 
features  are  not  well  simulated.  In  the  assessment,  this  was  evident  in  some  of  the  more  rugged 
runs  in  which  the  behavior  of  the  vehicle  in  the  field  exceeded  behaviors  in  RIVET. 

3.2  Path  Blockages 

In  the  synthetic  environment  of  RIVET,  XUV  routes  were  blocked  by  inserting  barrel  entities 
into  the  simulation  environment  at  the  desired  location  across  the  width  of  the  trail.  This 
provided  an  effective  path  blockage,  as  illustrated  in  figure  6. 


Figure  6.  Scenario  5  cul-de-sac  with  blocked  trails,  as  seen  in  RIVET. 
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In  the  field,  creating  a  blocked  path  for  the  XUV  was  considerably  more  difficult.  Vehicles, 
barrels,  and  cones  were  needed  to  fully  block  the  path  along  the  trails.  Any  area  of  sparse 
shrubs/trees  was  also  augmented  with  man-made  obstacles  to  achieve  the  intended  blockage 
through  an  area. 

Figure  7  is  a  panoramic  photo  of  the  actual  area  simulated  in  RIVET,  where  the  trail  on  the  left  is 
blocked  with  barrels  and  cones,  the  low  brush  in  the  middle  is  blocked  with  cones,  and  the  trail 
on  the  right  is  blocked  with  a  high-mobility  multipurpose  wheeled  vehicle  (HMMWV).  One  can 
easily  see  the  difference  (compare  to  figure  6)  between  RIVET  and  the  actual  area  in  the 
representation  of  the  trees  and  shrubs. 


Figure  7.  Scenario  5  cul-de-sac  with  blocked  trails. 


4.  Scenario  Background 


Fielded  UGVs  will  operate  in  an  almost  infinite  variety  of  terrain  conditions.  With  the  terrain 
available  in  Tactical  Vehicle  Maneuver  Area-Bravo  (TMVA-B)  at  FTIG,  seven  scenarios  were 
devised  on  four  separate  areas  of  terrain.  The  objective  in  defining  scenarios  was  to  construct 
diverse  situations  that  would  defeat  the  AM  capability  (cause  the  XUV  to  back  three  times  and 
call  for  operator  assistance)  but  still  allow  the  potential  for  the  DR  and  the  FCI  planner  to  find, 
without  operator  intervention,  an  alternate  route  to  the  goal. 

Scenarios  were  constructed  using  the  following  approach.  In  each  scenario,  an  SP  and  goal  were 
selected  such  that  given  the  a  priori  data,  the  global  planner  could  generate  a  straightforward  and 
easily  achievable  initial  plan  over  the  given  terrain.  Hundreds  of  XUV  field  runs  previously 
conducted  in  this  program  had  shown  that  unobstructed  paths  were  normally  completed  with  the 
AM  planner.  Then,  to  challenge  all  three  mobility  planners,  each  route  was  physically  blocked 
to  make  XUV  passage  impossible  along  the  intended  route.  The  blockage  was  not  available  to 
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the  global  map  and  hence  the  initial  plan,  but  rather  had  to  be  locally  sensed  by  the  XUV.  The 
blocked  passage  was  expected  to  force  the  planners  to  search  for  an  alternate  route  to  the  goal. 
Each  scenario  and  terrain  area  had  unique  challenges  that  are  described  in  the  following  section, 
A  limited  use  of  a  priori  terrain  information  was  provided  to  help  shape  each  situation.  In  one 
scenario,  false  a  priori  data  was  introduced  to  suggest  a  likely  alternate  path  that,  if  visited, 
would  also  be  found  impassible  by  the  XUV. 


5.  Experimental  Approach 


5.1  Experimental  Design 

An  experimental  design  was  constructed  that  could  be  executed  both  in  simulations  and  in  field 
runs  at  FTIG.  This  design  included  seven  scenarios  and  three  planners.  A  single  replication  of  a 
full  factorial  design,  crossing  each  scenario  with  each  planner,  would  require  21  runs.  The  intent 
of  the  experimental  design  was  to  provide  a  span  of  cul-de-sac  challenges  for  the  planners  that 
would  fairly  stress  the  new  technology.  Absolute  measurement  of  performance  in  operational  or 
field  testing  environments  is  elusive.  Often,  there  is  no  single  quantitative  measure,  such  as  time 
or  distance,  that  adequately  represents  performance.  Did  the  planner  allow  the  XUV  to 
overcome  the  specific  blockage  challenge  and  was  navigation  to  the  goal  point  achieved?  These 
questions  are  answered  with  an  affirmative  or  negative. 

To  gain  additional  insight,  expectations  remained  subjective  on  the  performance  of  the  XUV  in 
the  field.  Subject  matter  experts  (SMEs)  evaluated  the  XUV  behavior  at  various  points  along  a 
course  according  to  a  relatively  common  expectation  of  performance.  These  expectations  were 
forged  in  discussion  prior  to  the  experiment  and  helped  determine  the  seven  routes  chosen.  The 
expectations,  prior  to  the  field  testing,  were  further  solidified  through  a  simulation  study  working 
toward  a  reference  distribution  of  behavior  for  the  XUV  under  the  three  planners  and  seven 
courses.  Performance  expectation  is  where  the  analysis  starts  for  each  scenario. 

5.2  Parametric  Study  in  Simulation 

All  seven  scenarios  were  modeled  in  the  RIVET  environment,  and  a  parametric  study  consisting 
of  63  runs  was  conducted  in  RIVET.  Variables  were  scenario  (seven  levels)  and  planners  (AM, 
DR,  FCI)  with  three  repetitions  of  each  configuration.  Tree/shrub  density  and  distribution  were 
not  varied  as  additional  conditions  because  of  time  constraints;  although,  it  was  conjectured  that 
doing  so  may  have  introduced  more  realistic  variation  in  the  simulated  outcomes.  The  RIVET 
runs  were  used  to  verify  the  performance  of  the  planning  algorithms  as  well  as  validate  the 
experimenter’s  expectations  of  the  planners’  performance  in  each  scenario.  All  63  runs  were 
executed  in  RIVET.  The  results  of  the  RIVET  runs  are  included  in  the  following  discussion. 
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5.3  Field  Experiment 


The  field  experiment  was  designed  to  replieate  the  RIVET  parametric  study  but  with  additional 
replications  because  of  expected  variations  in  the  XUV  performance  in  the  field  environment. 
More  replications  would  be  required  for  trends  in  performance  to  become  evident.  The  variables 
were  scenario  (seven  levels),  planner  (three  levels),  and  repetitions  (five  reps  in  the  field).  To 
save  time  and  accomplish  as  many  useful  runs  as  possible,  not  all  replications  in  each  cell  were 
executed.  For  the  AM  runs,  when  it  became  apparent  that  repeated  runs  using  AM  would 
produce  identical  failed  results,  fewer  replications  were  done  with  that  planner  (see  table  1). 

Table  1 .  The  number  of  runs  executed  in  RIVET  and  in  the  field. 


Scenario 

No.  of  Runs 

RIVET 

Field 

AM 

DR 

FCI 

AM 

DR 

FCI 

1 

3 

3 

3 

1 

5 

3 

2 

3 

3 

3 

4 

4 

4 

3 

3 

3 

3 

2 

4 

3 

4 

3 

3 

3 

— 

— 

— 

5 

3 

3 

3 

2 

5 

5 

6 

3 

3 

3 

1 

5 

5 

7 

3 

3 

3 

5 

5 

5 

Scenario  4,  a  variant  of  scenario  3  with  additional  a  priori  data,  was  not  run  in  the  field  because  it 
was  believed  that  the  slight  difference  in  a  priori  data  between  scenarios  3  and  4  would  not 
appreciably  affect  XUV  performance  for  any  planner. 

In  scenarios  5  and  6,  a  successful  solution  of  the  cul-de-sac  and  subsequent  travel  to  the  goal 
resulted  in  a  run  of -1800  m.  To  conserve  experimental  time,  the  runs  were  truncated  after  the 
XUV  completely  exited  the  cul-de-sac  area  in  the  “finger”  and  moved  to  the  tank  trail  and  a  clear 
path  to  the  goal. 

Several  runs  were  rerun  because  of  events  unrelated  to  the  performance  of  the  planners.  For 
example,  a  steering  component  was  broken  on  one  run,  and  in  another  run,  unmapped  water  from 
a  recent  rain  (not  near  the  cul-de-sac)  created  a  mobility  barrier  that  caused  an  emergency  stop 
(e-stop)  for  safety. 

5.4  Data  Collected 

The  data  collected  included  configuration  files,  XUV  text  logs  (csv  and  bin  formats),  observer 
comments,  and  bufMor  (internal  AM  logs)  files  for  each  run.  Vehicle  logs  from  the  XUV 
included  time,  easting,  northing,  status  messages  from  XUV  movement,  and  planned  paths 
(easting  and  northing)  from  the  global  planner.  The  XUV  logs  are  the  basic  data  analyzed  for 
this  report. 
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5.5  Data  Reduction 

XUV  logs  were  converted  to  Excel  files,  and  duplicate  plan  entries  were  eliminated.  For  DR 
runs,  the  global  planner  generated  and  passed  to  the  XUV  more  plans  than  the  XUV  could 
execute.  Newly  generated  consecutive  plans  were  compared,  and  if  the  new  plan  was  within  5  s 
and  5  m  in  distance  of  the  current  plan,  it  was  removed.  This  more  accurately  described  the 
actual  XUV  performance. 

Following  the  completion  of  the  experiment,  notes  from  five  observers  were  compared  to 
establish  which  runs  would  be  included  in  the  analysis. 

5.6  Graphics  for  Each  Run 

Graphics  illustrating  the  XUV  path  and  narratives  describing  each  run  are  located  in  appendix  A. 
Experimenters’  observations  are  located  in  appendix  B.  Figure  8  is  an  example  of  the  run 
graphics. 


Figure  8.  Sample  graphic  from  run  76. 
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6.  Scenario  1 


6.1  Situation 

In  scenario  1,  the  SP  was  on  the  trail  next  to  the  tree  line  on  Dan  Grove  Highway  (figure  9).  The 
goal  was  ~300  m  northeast  of  the  start.  A  priori  data  included  a  trail  from  the  SP  to  the  goal, 
trees  to  the  west  of  the  trail,  an  a  priori  trail  to  the  goal  beyond  the  blockage,  and  a  nuclear, 
biological,  and  chemical  (NBC)  area  (XUV  no  go)  along  the  main  tank  trail  to  the  east.  The 
NBC  area  was  to  prevent  the  XUV  from  taking  the  low-cost  route  along  a  main  trail, 
encouraging  it  to  consider  another  alternative  route  instead.  The  trail  was  blocked  ~60  m  from 
the  SP  in  an  area  surrounded  by  trees  and  high  brush.  There  was  not  room  for  the  XUV  to  turn 
around  at  the  point  of  blockage.  Backing  up  was  the  only  possible  maneuver. 


Figure  9.  Scenario  1  description. 


6.2  Expectations 

The  expectations  were  that  (1)  AM  would  back  up  three  times  and  need  operator  assistance,  and 
(2)  upon  seeing  the  blockage,  the  DR  and  PCI  would  replan  back  up  the  trail  and  look  for  an 
opening  to  the  east  where  an  unmarked  trail  paralleled  the  original  route  and  was  open  to  the 
goal;  potentially,  on  the  DR  runs,  HMP  would  enable  the  XUV  to  turn  around  in  this  confined 
space.  FCI  would  need  to  back  up  to  the  new  planned  path  because  of  the  lack  of  HMP 
interface. 
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6.3  Results 


The  following  list  describes  the  column  headings  in  the  results  tables: 

•  Scenario:  1-7. 

•  Planner:  AM,  DR,  FCl. 

•  Source:  RIVET  or  field. 

•  Run:  run  number. 

•  Plans:  number  of  unique  global  plans  found  in  the  XUV  log  file. 

•  Distance:  distance  XUV  traveled  in  meters. 

•  Total  time:  time  from  start  to  stop  in  minutes. 

•  Time  eng.  off:  time  the  engine  was  off  (primarily  from  stalls). 

•  Time  HMP:  time  spent  with  HMP  as  the  local  planner. 

•  Av  spd  kph:  average  speed  entire  run  (dist/[total  time  -  time  eng.  off]). 

•  ET:  elapse  time  (total  time  -  time  eng  off). 

•  Outcome:  completed  or  DNF  (did  not  finish). 

•  Backups:  number  of  times  XUV  backed  up. 


Table  2.  Scenario  1  results. 


Scenario 

# 

Ranner 

Source 

Run 

Hans 

Dist  (m) 

Total  time 
(min) 

eng_off 

(min) 

Time 

HMP 

(min) 

ET  (min) 

avg  spd 
kph 

Backups 

Outcome 

1 

AM  only 

Field 

33 

156.19 

3.17 

3.17 

2.96 

3 

Stuck 

1 

AM  only 

RIVET 

238 

151.54 

2.40 

0.00 

2.40 

3.79 

3 

Backups 

1 

AM  only 

RIVET 

239 

152.71 

2.59 

0.00 

2.59 

3.53 

3 

Backups 

1 

AM  only 

RIVET 

240 

155.13 

2.39 

0.00 

2.39 

3.90 

3 

Backups 

1 

DR 

Field 

31.3 

20 

311.68 

8.18 

4.06 

8.18 

2.29 

Complete 

1 

DR 

Field 

35 

15 

369.98 

17.56 

5.53 

17.56 

1.26 

4 

Complete 

1 

DR 

Field 

38 

2 

178.86 

6.83 

3.33 

6.83 

1.57 

6 

Stuck 

1 

DR 

Field 

42 

13 

520.49 

10.22 

0.50 

6.69 

9.73 

3.21 

5 

Complete 

1 

DR 

Field 

43 

5 

134.06 

5.63 

1.91 

5.63 

1.43 

1 

HMP 

1 

DR 

RIVET 

231 

10 

257.82 

3.90 

0.00 

3.90 

3.90 

3.97 

Complete 

1 

DR 

RIVET 

232 

13 

261.07 

3.19 

0.00 

3.16 

3.19 

4.91 

1 

Complete 

1 

DR 

RIVET 

233 

10 

236.94 

2.92 

0.00 

2.92 

2.92 

mm 

1 

FCl 

Field 

32 

1 

317.28 

4.12 

4.12 

mm 

3 

1 

FCl 

Field 

34 

1 

169.87 

2.85 

2.85 

3.57 

3 

Backups 

1 

FCl 

Field 

39 

1 

168.56 

2.66 

0.37 

2.30 

4.40 

3 

Backups 

1 

FCl 

RIVET 

226 

1 

155.77 

2.27 

0.00 

2.27 

4.12 

3 

Backups 

1 

FCl 

RIVET 

227 

1 

141.00 

2.15 

2.15 

3.93 

3 

Backups 

1 

FCl 

RIVET 

228 

1 

156.28 

2.24 

2.24 

4.18 

3 

Backups 
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6.4  Autonomous  Mobility 

The  three  RIVET  runs  with  AM  planner  resulted  in  identical  behaviors.  The  XUV  moved  down 
the  trail,  saw  the  blockage,  backed  up  three  times,  and  the  run  was  halted. 

One  field  run  (run  33)  with  AM  was  conducted  and  similarly  ended  after  max  backups.  AM  was 
unable  to  find  an  alternate  route. 

6.5  Dynamic  Replanner 

The  three  RIVET  runs  with  the  DR  resulted  in  identical  behaviors.  The  XUV  moved  down  the 
trail  and  saw  the  blockage.  HMP  maneuvered  the  XUV  back  to  the  point  where  an  opening 
existed  to  the  southeast.  The  XUV  then  drove  through  the  opening  and  discovered  an  unmarked 
trail  to  its  left  and  parallel  to  the  original  plan.  It  followed  that  trail  to  the  goal.  In  the  RIVET 
runs,  HMP  was  invoked  99.7%  of  the  time  on  all  three  runs.  All  three  RIVET  runs  achieved  the 
goal.  The  orange  triangles  appearing  in  the  figures  of  DR  runs  (figures  A-3-A-8)  represent  the 
location  of  the  XUV  when  a  new  global  plan  to  the  goal  is  accepted  by  the  XUV.  Hence,  many 
new  plans  are  observed  in  cul-de-sac  situations,  and  fewer  new  plans  are  observed  in  less 
congested  environments  where  the  XUV  is  moving  easily  toward  the  goal. 

Three  of  the  five  field  runs  achieved  the  goal.  Two  of  the  three  successful  field  runs,  runs  31.3 
and  42,  closely  resembled  the  RIVET  runs.  For  each,  the  XUV  saw  the  blockage,  returned  to  the 
opening  on  the  southeast,  discovered  the  unmarked  trail  to  its  left,  and  proceeded  slowly  down 
the  unmarked  trail  under  HMP  control.  The  DR  continuously  generated  plans  that  directed  the 
XUV  back  to  the  original  trail,  beyond  the  blockage.  However,  the  XUV  was  unable  to  find  a 
clear  path  through  the  brush  and  trees,  and  eventually  the  DR  gave  a  plan  that  followed  the 
unmarked  trail  to  the  goal.  On  run  35,  the  XUV  followed  a  similar  path  but  this  time  found  a 
clear  path  back  to  the  original  trail  and  completed  the  run  on  the  original  trail  past  the  blockage. 
Runs  38  and  43  ended  in  e-stops  when  the  HMP  planner  backed  the  XUV  into  an  unknown  area 
of  dense  brush. 

6.6  Field  Cost  Interface 

Under  RIVET,  all  three  FCl  runs  went  as  expected.  None  achieved  the  goal  point.  All  FCI  runs 
in  RIVET  saw  the  blockage,  backed  up  three  times,  and  the  runs  were  halted. 

Field  runs  34  and  39  were  very  similar  to  RIVET  runs,  stopping  after  three  backups.  On  run  32, 
the  XUV  backed  up  three  consecutive  times  without  moving  forward.  (More  typical  of  the 
backup  behavior  is  for  the  XUV  to  move  forward  and  probe  the  local  terrain  after  changing  the 
perspective  via  the  backup.)  This  combined  60-m  backup  enabled  it  to  discover  the  opening  to 
the  southeast  and  the  unmarked  trail  paralleling  the  original  path  to  the  goal.  The  XUV 
proceeded  down  the  unmarked  trail  to  the  goal. 
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7.  Scenario  2 


7.1  Situation 

Scenario  2  is  a  derivation  of  scenario  1  with  the  same  SP  and  goal  (figure  10).  The  blockage  was 
placed  further  north  along  the  trail,  after  an  opening  to  an  alternate  trail  to  the  left  that  had  a  path 
to  the  goal.  A  priori  data  for  scenario  2  is  the  same  as  scenario  1 .  There  was  not  room  for  the 
XUV  to  turn  around  at  the  point  of  blockage.  Backing  up  was  the  only  possible  maneuver. 


Figure  10.  Scenario  2  description. 


7.2  Expectations 

The  expectations  were  that  (1)  AM  would  back  up  three  times  and  need  operator  assistance, 
ending  the  run  for  this  experiment,  and  (2)  upon  seeing  the  blockage,  the  DR  and  FCI  would 
replan  back  the  trail  and  look  for  an  opening  to  the  west  where  a  trail  branched  north  from  the 
original  planned  route  and  was  open  to  the  goal.  Potentially,  HMP  would  enable  the  XUV  to 
turn  around  in  this  confined  space,  while  FCI  would  need  to  back  up  to  the  new  planned  path 
because  of  the  lack  of  HMP  interface. 
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7.3  Results 


Table  3.  Scenario  2  results. 


Scenario 

# 

Ranner 

Source 

Run 

Rans 

Dist  (m) 

Total  time 
(min) 

eng_off 

(min) 

Time  HMP 
(min) 

ET  (min) 

avg  spd 
kph 

Backups 

Outcome 

2 

AM  only 

Field 

47 

257.15 

3.11 

0.27 

2.84 

5.42 

3 

Complete 

2 

AM  only 

Field 

51 

251.04 

7.27 

0.02 

7.27 

2.07 

4 

Complete 

2 

AM  only 

Field 

52 

146.00 

1.71 

1.71 

5.12 

Complete 

2 

AM  only 

Field 

57 

91.19 

2.21 

2.21 

2.47 

2 

E-stop 

2 

AM  only 

RIVET 

241 

189.70 

2.80 

0.00 

2.80 

4.07 

3 

Backups 

2 

AM  only 

RIVET 

242 

182.28 

2.54 

0.00 

2.54 

4.31 

3 

Backups 

2 

AM  only 

RIVET 

243 

185.03 

2.55 

0.00 

2.55 

4.36 

3 

Backups 

2 

DR 

Field 

48 

7 

273.20 

4.97 

0.38 

2.26 

4.59 

3.57 

2 

Complete 

2 

DR 

Field 

49 

7 

368.24 

8.47 

5.80 

8.47 

2.61 

2 

Complete 

2 

DR 

Field 

53 

4 

201.70 

2.53 

0.36 

0.94 

2.17 

5.57 

1 

Complete 

2 

DR 

Field 

56 

5 

91.32 

3.01 

0.20 

0.91 

2.81 

1.95 

E-Stop  HMP 

2 

DR 

RIVET 

206 

5 

326.57 

4.15 

0.00 

3.16 

4.15 

4.73 

Complete 

2 

DR 

RIVET 

207 

6 

328.16 

4.94 

0.00 

3.93 

4.94 

3.98 

Complete 

2 

DR 

RIVET 

208 

6 

345.55 

5.90 

0.00 

5.05 

5.90 

3.51 

Complete 

2 

FCI 

Field 

46 

1 

189.29 

5.52 

5.52 

2.06 

4 

Backups 

2 

FCI 

Field 

50 

1 

394.78 

8.37 

8.37 

2.83 

6 

Complete 

2 

FCI 

Field 

54 

1 

132.78 

1.61 

1.61 

4.95 

Complete 

2 

FCI 

Field 

55 

1 

174.75 

4.69 

4.69 

2.24 

4 

E-stop 

2 

FCI 

RIVET 

220 

1 

180.26 

2.27 

0.00 

2.27 

4.77 

2 

Backups 

2 

FCI 

RIVET 

221 

1 

181.82 

2.46 

0.00 

2.46 

4.44 

3 

Backups 

2 

FCI 

RIVET 

222 

1 

243.02 

3.43 

0.00 

3.43 

4.25 

5 

Backups 

7.4  Autonomous  Mobility 

The  three  RIVET  runs  with  AM  planner  resulted  in  identical  behaviors.  The  XUV  moved  down 
the  trail,  saw  the  blockage,  and  backed  up  three  times.  The  runs  were  ended  after  three 
consecutive  backups.  AM  was  unable  to  find  an  alternate  route. 

Three  of  the  four  AM  field  runs  (47,  51,  and  52)  were  able  to  successfully,  but  unexpectedly, 
find  an  alternate  route  to  the  east  of  the  blockage.  This  was  probably  the  result  of  sparse  enough 
brush  in  the  area  for  AM  to  find  a  local  path  and  move  the  XUV  through  to  the  goal.  With  each 
successful  run  through  the  brush,  the  next  run  became  easier  for  the  planner  to  find  a  path 
through  the  brush.  Run  57  ended  in  an  e-stop  when  the  XUV  backed  too  close  to  a  HMMWV 
used  for  blockage. 

None  of  the  field  runs  for  AM  were  similar  to  the  RIVET  runs. 

7.5  Dynamic  Replanner 

The  three  RIVET  runs  with  the  DR  resulted  in  identical  behaviors.  The  XUV  moved  down  the 
trail  and  saw  the  blockage.  HMP  maneuvered  the  XUV  back  to  the  point  where  an  opening 
existed  to  a  trail  to  the  east.  The  XUV  then  followed  the  trail  to  the  goal.  In  the  RIVET  runs, 
HMP  was  invoked  76%-86%  of  the  time. 
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Three  of  the  four  field  runs  achieved  the  goal.  On  runs  48,  49,  and  53,  the  XUV  saw  the 
blockage,  generated  new  plans  to  the  east  of  the  blockage,  and  found  its  way  to  the  goal.  No 
plans  were  generated  on  these  runs  that  would  have  utilized  the  trail  to  the  west,  leading  to 
speculation  that  the  alternate  trail  was  not  in  the  a  priori  data  as  it  was  in  RIVET.  As  with  AM, 
the  more  trips  through  the  shrubs,  the  wider  and  easier  the  path  became.  It  is  unclear  whether  the 
NBC  area  intended  as  part  of  the  a  priori  data  was  set  for  all  runs.  These  three  successful  runs, 
upon  taking  the  alternative  path  to  the  east,  continued  east  to  a  main  tank  trail  that  should  have 
been  avoided  due  to  an  NBC  presence.  In  run  56,  the  XUV  was  e-stopped  after  two  backups 
because  it  backed  into  dense  shrubs. 

None  of  the  field  runs  for  DR  resembled  the  RIVET  runs. 

7.6  Field  Cost  Interface 

None  of  the  three  RIVET  runs  achieved  the  goal  point;  all  RIVET  runs  ended  after  three  backups 
shortly  after  encountering  the  blockage.  Again,  this  was  unexpected  as  there  should  have  been  a 
path  to  the  west  in  the  a  priori  data. 

Runs  50  and  54  achieved  the  goal.  These  runs  were  similar  to  the  successful  DR  and  AM  runs 
because  the  XUV  saw  the  blockage,  moved  off  the  trail  to  the  east  of  the  blockage  through  high 
shrubs,  and  found  its  way  to  the  goal.  As  with  previous  runs,  the  more  trips  through  the  shrubs, 
the  wider  and  easier  the  path  became.  Run  46.2  was  e-stopped  because  of  max  backups.  Run 
55.1  was  e-stopped  because  it  backed  too  close  to  the  HMMWV  used  in  the  blockage. 

One  field  run  for  FCI  was  similar  to  the  three  unsuccessful  RIVET  runs. 


8.  Scenario  3 


8.1  Situation 

In  scenario  3,  the  SP  was  on  the  trail  at  the  base  of  a  hill  to  the  north  of  the  helicopter  pad  (figure 
11).  The  goal  was  on  the  helicopter  pad,  and  there  was  an  open  trail  from  the  SP  up  the  hill 
directly  to  the  goal.  The  trail  was  deeply  rutted,  steep,  and  surrounded  by  high  shrubs  on  each 
side,  and  was  included  in  the  a  priori  data.  An  alternate,  unmarked  trail  existed  to  the  west  of  the 
original  path  and  a  “false”  trail  to  the  east  of  the  original  plan  was  included  in  the  a  priori  data. 
This  false  a  priori  trail  was  not  passable  by  the  XUV.  The  planned  path  was  blocked  ~80  m  from 
the  SP  on  the  steepest  part  of  the  deeply  rutted  trail  with  high  shrubs  on  both  sides.  There  was 
not  room  for  the  XUV  to  turn  around  at  the  point  of  blockage.  Backing  up  was  the  only  possible 
maneuver.  The  only  possible  path  to  the  goal  was  an  unmarked  trail  to  the  west  of  the  original 
plan. 
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Figure  1 1 .  Scenario  3  description. 


8.2  Expectations 

The  expectations  were  that  (1)  AM  would  back  up  three  times  and  need  operator  assistance,  and 
(2)  upon  seeing  the  blockage,  the  DR  and  FCI  would  replan  back  the  trail  and  look  for  an 
opening  to  the  east  where  an  a  priori  trail  gave  a  path  to  the  goal,  discover  that  this  way  was 
impassible,  and  then  search  for  a  path  to  the  west  of  the  original  plan,  hopefully  finding  the 
unmarked  trail  to  the  goal. 

8.3  Results 

Table  4.  Scenario  3  results. 


Scenario 

# 

Hanner 

Source 

Run 

Rans 

Dist  (m) 

Total  time 
(min) 

eng_off 

(min) 

Time 

HMP 

(min) 

ET  (min) 

avg  spd 
kph 

Backups 

Outcome 

3 

AM  only 

Field 

63 

210.30 

2.89 

0.25 

2.65 

4.77 

3 

Backups 

3 

AM  only 

Field 

64 

293.85 

3.67 

0.23 

3.44 

5.13 

3 

Backups 

3 

AM  only 

RIVET 

293 

1 

142.57 

1.87 

0.00 

1.87 

4.57 

3 

Backups 

3 

AM  only 

RIVET 

294 

1 

144.37 

1.97 

0.00 

1.97 

4.39 

3 

Backups 

3 

AM  only 

RIVET 

295 

1 

142.29 

2.01 

0.00 

2.01 

4.25 

3 

Backups 

3 

DR 

Field 

62 

8 

139.31 

18.35 

10.52 

18.35 

0.46 

1 

HMP 

3 

DR 

Field 

65 

19 

446.51 

12.97 

9.51 

12.97 

2.07 

3 

E-stop 

3 

DR 

Field 

68 

21 

187.07 

6.54 

0.26 

5.57 

6.28 

1.79 

HMP 

3 

DR 

Field 

72 

3 

95.02 

0.67 

0.02 

0.17 

0.65 

8.73 

E-stop 

3 

DR 

RIVET 

290 

6 

253.34 

2.60 

0.00 

1.95 

2.60 

5.85 

Complete 

3 

DR 

RIVET 

291 

8 

241.73 

2.65 

0.00 

2.05 

2.65 

5.48 

Complete 

3 

DR 

RIVET 

292 

8 

282.77 

3.64 

0.00 

2.06 

3.64 

4.67 

2 

Complete 

3 

FCI 

Field 

61 

1 

310.97 

4.65 

4.65 

4.02 

3 

Complete 

3 

FCI 

Field 

66 

1 

268.25 

4.30 

0.47 

3.83 

4.21 

6 

Backups 

3 

FCI 

Field 

69 

1 

339.58 

4.94 

0.25 

4.69 

4.35 

4 

Complete 

3 

FCI 

RIVET 

286 

1 

959.97 

15.01 

0.00 

15.01 

3.84 

27 

Backups 

3 

FCI 

RIVET 

287 

1 

201.96 

2.35 

0.00 

2.35 

5.15 

3 

Backups 

3 

FCI 

RIVET 

288 

1 

519.61 

8.07 

0.00 

8.07 

3.86 

13 

Backups 
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8.4  Autonomous  Mobility 

The  three  RIVET  runs  with  AM  planner  resulted  in  identical  behaviors.  The  XUV  moved  up  the 
trail,  saw  the  blockage,  and  backed  up  three  times.  The  runs  were  ended  after  three  consecutive 
backups.  AM  was  unable  to  find  an  alternate  route. 

The  two  field  runs  with  AM  planner  (63  and  64)  resulted  in  behavior  identical  to  the  RIVET 
runs.  The  XUV  moved  up  the  trail,  saw  the  blockage,  and  backed  up  three  times.  The  trail  at  the 
point  of  the  blockage  was  steep  and  deeply  rutted,  making  any  maneuver  difficult.  The  runs 
were  ended  after  three  consecutive  backups.  AM  was  unable  to  find  an  alternate  route. 

8.5  Dynamic  Replanner 

The  three  RIVET  runs  with  the  DR  resulted  in  identical  behaviors  (figure  12).  The  XUV  moved 
up  the  trail  and  saw  the  blockage.  HMP  maneuvered  the  XUV  back  to  the  point  where  an 
opening  existed  to  the  west.  DR  then  generated  successive  plans  to  move  the  XUV  to  the  goal 
through  an  unmarked  trail.  The  a  priori  road  to  the  east  was  not  explored.  In  the  RIVET  runs, 
HMP  was  invoked  57%  to  77%  of  the  time. 


Figure  12.  RIVET  run  290. 

None  of  the  four  field  runs  achieved  the  goal.  On  runs  62  and  68,  the  XUV  saw  the  blockage  and 
generated  new  plans  back  to  the  point  where  an  opening  existed  to  the  west.  However,  HMP  was 
unable  to  maneuver  the  XUV  back  down  the  steep  trail,  and  both  runs  were  e-stopped  when  HMP 
backed  into  high  brush.  During  run  72,  the  XUV  went  through  the  barrels  at  the  blockage  and  was 
e-stopped.  On  run  65,  the  XUV  maneuvered  back  to  the  base  of  the  hill  and  explored  the  false 
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a  priori  trail  to  the  east,  finding  it  impassable.  It  then  generated  plans  to  the  west  of  the  original 
plan  toward  the  unmarked  trail  to  the  goal.  However,  the  XUV  failed  to  find  the  small  opening 
in  the  brush  for  the  unmarked  trail  and  was  e-stopped  short  of  the  goal.  In  the  field  runs,  HMP 
was  invoked  25%  to  85%  of  the  time. 

None  of  the  field  runs  resembled  the  RIVET  runs. 

8.6  Field  Cost  Interface 

The  three  RIVET  runs  with  the  FCI  planner  resulted  in  identical  behaviors.  The  XUV  moved  up 
the  trail,  saw  the  blockage,  and  backed  up  three  times  until  the  run  was  halted  for  max  backups. 

Two  of  the  three  FCI  field  runs  achieved  the  goal.  On  runs  61  and  69,  the  XUV  saw  the 
blockage,  backed  to  the  base  of  the  hill,  and  immediately  explored  the  area  to  the  west  of  the 
original  plan,  discovering  the  unmarked  trail  to  the  goal.  In  run  66,  the  XUV  moved  up  the  trail, 
saw  the  blockage,  and  backed  up  six  times  until  the  run  was  halted  for  max  backups. 


9.  Scenario  4 


9.1  Situation 


Scenario  4  was  exactly  like  scenario  3,  with  the  exception  of  additional  a  priori  data  on  trees  and 
shrubs  near  the  alternate  unmarked  trail  that  existed  to  the  west  of  the  original  path  (figure  13). 

It  was  thought  that  additional  a  priori  data  may  increase  the  probability  of  finding  the  unmarked 
trail  to  the  goal. 


Figure  13.  Scenario  4  description. 
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9.2  Expectations 


The  expectations  were  that  (1)  AM  would  back  up  three  times  and  need  operator  assistance,  and 
(2)  upon  seeing  the  blockage,  the  DR  and  FCI  would  replan  back  the  trail  and  look  for  an 
opening  to  the  east  where  an  a  priori  trail  gave  a  path  to  the  goal,  discover  that  this  way  was 
impassible,  and  then  search  for  a  path  to  the  west  of  the  original  plan,  hopefully  finding  the 
unmarked  trail  to  the  goal  with  the  assistance  of  additional  a  priori  tree  data. 

9.3  Results 

Table  5.  Scenario  4  results. 


Scenario 

# 

Hanner 

Source 

Run 

Hans 

Dist  (m) 

Total  time 
(min) 

eng_off 

(min) 

Time  HMP 
(min) 

ET  (min) 

avg  spd 
kph 

Backups 

Outcome 

4 

AM  only 

RIVET 

283 

1 

143.09 

1.94 

0.00 

1.94 

4.43 

3 

Backups 

4 

AM  only 

RIVET 

284 

1 

145.30 

1.92 

0.00 

1.92 

4.54 

3 

Backups 

4 

AM  only 

RIVET 

285 

1 

136.00 

2.00 

0.00 

2.00 

4.08 

3 

Backups 

4 

DR 

RIVET 

280 

9 

241.34 

2.39 

0.00 

1.69 

2.39 

6.05 

Complete 

4 

DR 

RIVET 

281 

6 

247.78 

2.41 

0.00 

1.88 

2.41 

6.16 

Complete 

4 

DR 

RIVET 

282 

9 

238.60 

2.28 

0.00 

1.50 

2.28 

6.29 

Complete 

4 

FCI 

RIVET 

275 

1 

393.16 

5.52 

0.00 

5.52 

4.27 

9 

Backups 

4 

FCI 

RIVET 

276 

1 

201.17 

2.41 

0.00 

2.41 

5.01 

3 

Backups 

4 

FCI 

RIVET 

277 

1 

202.55 

2.34 

0.00 

2.34 

5.20 

3 

Backups 

9.4  Autonomous  Mobility 

Scenario  4  was  run  only  in  RIVET.  The  three  RIVET  runs  with  AM  planner  resulted  in  identical 
behaviors.  The  XUV  moved  up  the  trail,  saw  the  blockage,  and  backed  up  three  times.  The  runs 
were  ended  after  three  consecutive  backups.  AM  was  unable  to  find  an  alternate  route 

9.5  Dynamic  Replanner 

The  three  RIVET  runs  with  the  DR  resulted  in  identical  behaviors.  The  XUV  moved  up  the  trail 
and  saw  the  blockage.  HMP  maneuvered  the  XUV  back  to  the  point  where  an  opening  existed  to 
the  west.  DR  then  generated  successive  plans  to  move  the  XUV  to  the  goal  through  an 
unmarked  trail. 

9.6  Field  Cost  Interface 

The  three  RIVET  runs  with  the  FCI  planner  resulted  in  identical  behaviors.  The  XUV  moved  up 
the  trail,  saw  the  blockage,  and  backed  up  three  times  until  the  run  was  halted  for  max  backups. 
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10.  Scenario  5 


10.1  Situation 

In  scenario  5,  the  SP  was  at  the  opening  of  the  terrain  area  known  as  the  “finger”  in  the  “football 
field”  (figure  14).  The  goal  was  on  the  main  tank  trail  to  the  west  of  the  cul-de-sac,  ~400  m  west 
of  the  SP.  The  initial  plan  was  along  the  a  priori  trail  toward  the  goal,  which  narrowed  into  a 
one-lane  trail  for  -200  m,  then  ended  on  the  main  tank  trail  where  the  goal  was  about  50  m 
north.  A  priori  data  included  trails  from  the  SP  to  the  goal,  trees  to  the  north  and  south  of  the 
trail,  an  alternate  a  priori  trail  at  the  base  of  the  “finger”  north  of  the  original  path,  and  trees 
between  the  two  a  priori  trails.  The  planned  trail  was  blocked  where  the  trail  narrowed,  and  the 
alternate  a  priori  trail  was  also  blocked  where  it  entered  the  trees.  This  situation  constituted  a 
very  large,  open  cul-de-sac  with  plenty  of  maneuver  room  for  the  XUV. 


Figure  14.  Scenario  5  description. 

10.2  Expectations 

One  expectation  was  that  AM  would  see  the  blockage  and  repeatedly  circle,  trying  to  find  a  path 
through  the  original  plan.  For  this  reason,  a  “three  circle”  rule  was  instituted  to  end  a  run  if  the 
XUV  did  not  appear  to  be  finding  a  solution  after  circling  three  times  and/or  backing  up  three 
times  and  calling  for  operator  assistance.  Another  expectation  was  that  upon  seeing  the  initial 
blockage,  the  DR  and  FCI  would  replan  to  the  nearby  alternate  trail.  Upon  finding  it  blocked, 
DR  and  FCI  were  expected  to  plan  for  the  XUV  to  return  the  way  it  came  and  discover  an 
alternative  way  on  the  trail  around  the  trees  and  shrubs  to  the  east. 
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10.3  Results 


Table  6.  Scenario  5  results. 


Scenario 

# 

Ranner 

Source 

Run 

Rans 

Dist  (m) 

Total  time 
(min) 

eng_off 

(min) 

Time  HMP 
(min) 

ET  (min) 

avg  spd 
kph 

Backups 

Outcome 

5 

AM  only 

Field 

16 

265.76 

9.53 

0.32 

9.21 

1.73 

1 

circles 

5 

AM  only 

Field 

20 

451.59 

4.93 

4.93 

5.50 

2 

circles 

5 

AM  only 

RIVET 

234 

1 

248.93 

2.53 

0.00 

2.53 

5.91 

2 

Backups 

5 

AM  only 

RIVET 

235 

248.49 

2.47 

0.00 

2.47 

6.03 

3 

Backups 

5 

AM  only 

RIVET 

236 

293.74 

3.31 

0.00 

3.31 

5.33 

3 

Backups 

5 

DR 

Field 

18 

14 

653.39 

4.63 

2.15 

4.63 

8.47 

1 

Complete 

5 

DR 

Field 

19 

17 

824.49 

11.94 

4.92 

11.94 

4.14 

Complete 

5 

DR 

Field 

22 

8 

682.36 

4.92 

2.14 

4.92 

8.32 

Complete 

5 

DR 

Field 

27 

20 

727.25 

6.94 

0.32 

3.70 

6.62 

6.59 

Complete 

5 

DR 

Field 

30 

13 

757.55 

7.16 

2.02 

7.16 

6.35 

5 

Complete 

5 

DR 

RIVET 

23 

16 

635.43 

5.78 

0.00 

3.10 

5.78 

6.59 

Complete 

5 

DR 

RIVET 

24 

10 

689.19 

5.19 

0.00 

2.54 

5.19 

7.97 

Complete 

5 

DR 

RIVET 

201 

10 

627.00 

5.01 

0.00 

2.69 

5.01 

7.51 

1 

Complete 

5 

FCI 

Field 

17 

1 

336.93 

3.51 

0.07 

3.44 

5.87 

E-stop 

5 

FCI 

Field 

21 

1 

494.46 

9.79 

9.79 

3.03 

E-stop 

5 

FCI 

Field 

23 

1 

923.50 

12.36 

12.36 

4.48 

1 

Complete 

5 

FCI 

Field 

25 

2 

1248.00 

15.56 

15.56 

4.81 

10 

circles 

5 

FCI 

Field 

28 

1 

801.98 

5.53 

5.53 

8.71 

1 

Complete 

5 

FCI 

RIVET 

12 

1 

735.67 

5.73 

0.00 

5.73 

7.70 

2 

Complete 

5 

FCI 

RIVET 

13 

1 

725.60 

5.19 

0.00 

5.19 

8.38 

1 

Complete 

5 

FCI 

RIVET 

215 

1 

664.26 

4.87 

0.00 

4.87 

8.18 

1 

Complete 

10.4  Autonomous  Mobility 

All  three  RIVET  runs  were  ended  on  max  backups  immediately  upon  encountering  the  blockage 
on  the  initial  planned  path.  AM  would  see  the  blockage  of  the  original  path,  back  up,  and  be 
unable  to  find  a  plan  through  the  blockage. 

In  the  two  field  runs  (16.1  and  20)  with  the  AM  planner,  the  XUV  repeatedly  circled  in  the 
cul-de-sac,  trying  to  find  an  acceptable  way  along  the  planned  path.  After  circling  three  or  more 
times,  the  runs  were  e-stopped. 

10.5  Dynamic  Replanner 

In  the  three  RIVET  runs,  the  XUV  proceeded  along  the  planned  route,  saw  the  path  blocked,  and 
attempted  to  go  through  the  alternate  route  nearby  to  the  north  but  saw  that  it  also  was  blocked. 
The  XUV  then  replanned  back  east  past  the  starting  point,  around  the  a  priori  trees,  and  south 
toward  the  tank  trail  and  a  clear  path  to  the  goal.  These  runs  spent  49%-54%  of  the  time  in 
HMP  at  the  cul-de-sac  while  generating  and  trying  alternate  plans. 

The  five  field  runs  resembled  very  closely  the  RIVET  runs.  The  field  runs  tended  to  spend 
additional  time  and  distance  exploring  alternate  routes  in  the  cul-de-sac,  probably  because  of  the 
highly  varied  vegetation  on/around  the  alternate  paths  out  of  the  cul-de-sac.  These  runs  spent 
28%-53%  of  the  time  in  HMP  at  the  cul-de-sac  while  generating  and  trying  alternate  plans. 
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10.6  Field  Cost  Interface 


In  the  three  RIVET  runs,  the  XUV  preceded  along  the  planned  route,  saw  the  path  blocked,  and 
circled  several  times  while  attempting  to  find  a  way  through  the  original  and  alternate  trails.  The 
XUV  eventually  replanned  back  past  the  starting  point,  around  the  a  priori  trees,  and  south 
toward  the  tank  trail  and  a  clear  path  to  the  goal. 

Two  of  the  five  field  runs  were  successful  in  finding  an  alternate  route.  Runs  23  and  28  were 
successful  in  achieving  the  goal  but  were  100-200  m  longer  than  the  corresponding  RIVET  runs 
because  these  runs  spent  additional  time  and  distance  circling  in  the  cul-de-sac  and  building  up 
local  terrain  data  before  deciding  to  replan  back  past  the  starting  point,  around  the  a  priori  trees 
and  south  toward  the  tank  trail  and  a  clear  path  to  the  goal.  Three  of  the  runs  (21.1,  25.2,  and  17) 
saw  the  blockage  and  spent  additional  time  and  distance  circling  in  the  cul-de-sac,  but  FCI  never 
generated  a  plan  back  toward  the  SP.  The  runs  were  ended  after  three  or  more  circles. 

Two  of  the  five  field  runs  resembled  very  closely  the  RIVET  runs. 


11.  Scenario  6 


11.1  Situation 

Scenario  6  was  exactly  the  same  as  scenario  5  with  one  exception.  The  a  priori  tree  data  was 
shortened  by  ~  100  m  in  the  southerly  direction  (figure  15).  This  was  done  to  allow  the  XUV  to 
explore  for  alternate  paths  through  this  wooded  area,  which  had  several  unmarked  trails  through 
to  the  tank  trail  on  the  southern  side.  Plans  and  blockages  remained,  as  in  scenario  5. 


11.2  Expectations 


The  expectations  were  that  (1)  AM  would  see  the  blockage  and  repeatedly  circle,  trying  to  find  a 
path  through  the  original  plan,  and  (2)  upon  seeing  the  initial  blockage,  the  DR  and  FCI  would 
replan  to  the  nearby  alternate  trail.  Upon  finding  it  blocked,  DR  and  FCI  were  expected  to  plan 
for  the  XUV  to  return  the  way  it  came  and  discover  an  alternative  way  through  the  trees  and 
shrubs  to  the  south. 

11.3  Results 

Table  7.  Scenario  6  results. 


Scenario 

# 

Hanner 

Source 

Run 

Rans 

Dist  (m) 

Total  time 
(min) 

eng_off 

(min) 

Time 

HMP 

(min) 

ET  (min) 

avg  spd 
kph 

Backups 

Outcome 

6 

AM  only 

Field 

78 

480.19 

3.70 

3.70 

7.78 

1 

circles 

6 

AM  only 

RIVET 

261 

410.04 

4.92 

0.00 

4.92 

5.00 

5 

Backups 

6 

AM  only 

RIVET 

262 

452.09 

5.07 

0.00 

5.07 

5.35 

6 

Backups 

6 

AM  only 

RIVET 

263 

679.10 

9.40 

0.00 

9.40 

4.34 

12 

Backups 

6 

DR 

Field 

77.1 

20 

722.00 

19.69 

7.98 

19.69 

2.20 

4 

Complete 

6 

DR 

Field 

80 

18 

605.20 

7.15 

5.51 

7.15 

5.08 

Complete 

6 

DR 

Field 

84 

25 

719.32 

18.73 

5.79 

18.73 

2.30 

4 

Complete 

6 

DR 

Field 

86 

19 

557.95 

6.01 

3.07 

6.01 

5.57 

1 

Complete 

6 

DR 

Field 

89 

17 

560.73 

5.23 

3.10 

5.23 

6.43 

Complete 

6 

DR 

RIVET 

258 

15 

409.07 

3.20 

0.00 

1.36 

3.20 

7.67 

Complete 

6 

DR 

RIVET 

259 

14 

422.67 

3.10 

0.00 

1.25 

3.10 

8.18 

Complete 

6 

DR 

RIVET 

260 

15 

448.44 

3.82 

0.00 

1.52 

3.82 

7.05 

Complete 

6 

FCI 

Field 

76 

1 

896.50 

7.89 

7.89 

6.82 

Complete 

6 

FCI 

Field 

79.1 

1 

674.38 

5.27 

5.27 

7.68 

1 

Complete 

6 

FCI 

Field 

82 

1 

684.42 

4.66 

4.66 

8.81 

Complete 

6 

FCI 

Field 

85 

1 

1103.38 

9.69 

9.69 

6.83 

1 

Complete 

6 

FCI 

Field 

88 

1 

592.16 

3.90 

3.90 

9.11 

Complete 

6 

FCI 

RIVET 

251 

1 

511.35 

3.95 

0.00 

3.95 

7.76 

Complete 

6 

FCI 

RIVET 

254 

1 

589.00 

4.84 

0.00 

4.84 

7.30 

3 

Complete 

6 

FCI 

RIVET 

255 

1 

526.46 

3.87 

0.00 

3.87 

8.16 

1 

Complete 

11.4  Autonomous  Mobility 

All  three  RIVET  runs  ended  on  max  backups  after  circling  and  repeatedly  trying  to  go  through 
the  blocked  area. 

In  the  one  field  run  with  AM,  the  XUV  repeatedly  circled  in  the  cul-de-sac,  trying  to  find  an 
acceptable  way  along  the  initial  planned  path.  After  circling  three  or  more  times,  the  run  was 
stopped. 

11.5  Dynamic  Replanner 

In  the  three  RIVET  runs,  the  XUV  proceeded  along  the  planned  route,  saw  the  path  blocked,  and 
attempted  to  go  through  the  alternate  route  nearby  to  the  north  but  saw  that  it  also  was  blocked. 
The  XUV  then  replanned  back  past  the  starting  point  and  discovered  a  path  through  the  trees 
south  toward  the  tank  trail  and  a  clear  path  to  the  goal.  These  runs  spent  40%-42%  of  the  time 
in  HMP  at  the  cul-de-sac  while  generating  and  trying  alternate  plans. 
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The  five  field  runs  resembled  very  elosely  the  RIVET  runs.  The  field  runs  tended  to  spend 
additional  time  and  distanee  exploring  alternate  routes  in  the  cul-de-sae,  probably  because  of  the 
highly  varied  vegetation  on/around  the  alternate  paths  out  of  the  cul-de-sac.  These  runs  spent 
31%-77%  of  the  time  in  HMP  at  the  cul-de-sac  while  generating  and  trying  alternate  plans 

11.6  Field  Cost  Interface 

In  the  three  RIVET  runs,  the  XUV  proceeded  along  the  planned  route,  saw  that  the  path  was 
blocked,  and  circled  several  times  while  attempting  to  find  a  way  through  the  original  and 
alternate  trails.  The  XUV  eventually  replanned  back  past  the  starting  point  and  discovered  a 
route  through  the  trees  south  toward  the  tank  trail  and  a  clear  path  to  the  goal. 

All  five  field  runs  were  successful  in  finding  an  alternate  route  similar  to  the  runs  in  RIVET. 
These  runs  spent  additional  time  and  distance  circling  in  the  cul-de-sac,  building  up  local  terrain 
data  before  replanning  back  past  the  starting  point  and  discovering  a  path  through  the  trees  south 
toward  the  tank  trail  and  a  clear  path  to  the  goal. 

All  five  of  the  field  runs  resembled  very  closely  the  RIVET  runs. 


12.  Scenario  7 


12.1  Situation 

Scenario  7  was  the  simplest  scenario  (figure  16).  The  SP  and  goal  were  along  a  tree  line  in  the 
football  field.  The  trail  to  the  goal  went  between  a  solid  tree  line  to  the  north  and  a  small  set  of 
trees  and  shrubs  to  the  south.  Both  sets  of  trees  were  in  the  a  priori  data.  The  path  between  the 
trees  was  blocked  ~50  m  into  the  tree  line. 


Figure  16.  Scenario  7  description. 
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12.2  Expectations 


The  expectations  were  that  (1)  AM  would  see  the  blockage  and  repeatedly  back  up  because  of 
the  heavy  foliage  on  both  sides  of  the  trail  (it  was  expected  that  max  backups  would  end  these 
runs)  and  (2)  upon  seeing  the  initial  blockage,  the  DR  and  FCI  would  replan  back  to  the  split  in 
the  terrain  and  find  the  alternate  route  to  the  goal. 

12.3  Results 

Table  8.  Scenario  7  results. 


Scenario 

# 

Ranner 

Source 

Run 

Pans 

Dist  (m) 

Total  time 
(min) 

eng_off 

(min) 

Time  HMP 
(min) 

ET  (min) 

avg  spd 
kph 

Backups 

Outcome 

7 

AM  only 

Field 

2 

253.83 

3.34 

0.30 

3.05 

5.00 

2 

E-stop 

7 

AM  only 

Field 

4 

1 

341.02 

5.28 

5.28 

3.87 

3 

Complete 

7 

AM  only 

Field 

8 

206.87 

0.99 

0.99 

12.48 

Complete 

7 

AM  only 

Field 

10 

175.14 

2.71 

2.71 

3.88 

3 

E-stop 

7 

AM  only 

Field 

13 

1 

211.05 

2.91 

0.46 

2.44 

5.19 

3 

Max  Backups 

7 

AM  only 

RIVET 

248 

495.66 

3.45 

0.00 

3.45 

8.61 

Complete 

7 

AM  only 

RIVET 

249 

530.77 

3.79 

0.00 

3.79 

8.40 

Complete 

7 

AM  only 

RIVET 

250 

921.04 

8.01 

0.00 

8.01 

6.90 

2 

Complete 

7 

DR 

Field 

1 

11 

432.62 

7.37 

4.42 

7.37 

3.52 

5 

Complete 

7 

DR 

Field 

6 

8 

331.99 

6.11 

4.18 

6.11 

3.26 

Complete 

7 

DR 

Field 

7 

16 

368.12 

12.73 

5.73 

12.73 

1.73 

1 

E-stop 

7 

DR 

Field 

11 

13 

260.12 

19.40 

16.52 

19.40 

0.80 

3 

E-Stop  HMP 

7 

DR 

Field 

15 

19 

365.36 

9.62 

4.65 

9.62 

2.28 

6 

Max  Backups 

7 

DR 

RIVET 

212 

11 

282.32 

3.07 

0.00 

2.21 

3.07 

5.53 

Complete 

7 

DR 

RIVET 

213 

10 

296.26 

2.46 

0.00 

1.50 

2.46 

7.24 

Complete 

7 

DR 

RIVET 

214 

14 

281.37 

3.34 

0.00 

2.68 

3.34 

5.06 

Complete 

7 

FCI 

Field 

3 

1 

212.12 

2.85 

0.24 

2.62 

4.86 

3 

Max  Backups 

7 

FCI 

Field 

5 

1 

193.94 

2.52 

2.52 

4.62 

3 

Max  Backups 

7 

FCI 

Field 

9 

1 

193.41 

3.38 

0.24 

3.14 

3.69 

3 

Max  Backups 

7 

FCI 

Field 

12 

2 

186.52 

2.74 

0.24 

2.51 

4.47 

3 

Max  Backups 

7 

FCI 

Field 

14 

1 

213.63 

3.16 

3.16 

4.05 

3 

Max  Backups 

7 

FCI 

RIVET 

17 

1 

266.89 

1.49 

0.00 

1.49 

10.74 

Complete 

7 

FCI 

RIVET 

224 

1 

286.07 

2.00 

0.00 

2.00 

8.57 

Complete 

7 

FCI 

RIVET 

225 

1 

291.51 

2.06 

0.00 

2.06 

8.49 

Complete 

12.4  Autonomous  Mobility 

All  three  RIVET  runs  were  successfully  completed  as  the  XUV  sensed  the  blockage  and  turned 
around  180°  to  return  to  the  place  in  the  trail  where  an  alternative  trail  existed  to  the  goal.  The 
RIVET-modeled  foliage  around  the  blockage  allowed  this  maneuvering.  This  was  not  expected. 

The  field  runs  varied  considerably.  Run  8  completely  ignored  the  original  path  and  skirted  to  the 
south  of  the  cul-de-sac  directly  to  the  goal.  Run  4  found  the  cul-de-sac  but  backed  up  three  times 
and  was  then  positioned  in  a  location  where  it  was  able  to  find  an  open  path  along  the  trail  to  the 
west.  From  that  point,  it  went  directly  to  the  goal.  Run  2  was  e-stopped  at  the  blockage  site,  run 
10  was  e-stopped  when  the  XUV  backed  into  a  rock,  and  run  13  was  ended  for  max  backups. 
Although  two  field  runs  achieved  the  goal,  none  did  so  in  the  manner  of  RIVET,  where  the  XUV 
was  able  to  turn  in  the  simulated  environment. 
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12.5  Dynamic  Replanner 

In  the  three  RIVET  runs,  the  XUV  proceeded  along  the  planned  route,  saw  that  the  path  was 
blocked,  and  planned  a  way  out.  To  achieve  the  plan,  HMP  directed  a  180°  turn,  and  the  XUV 
followed  a  new  plan  back  the  trail  and  around  the  southern  side  of  the  trees  to  the  goal. 

In  the  field  runs,  after  the  blockage  was  encountered,  HMP  backed  the  XUV  the  entire  distance 
out  of  the  cul-de-sac  on  four  of  the  five  runs.  Two  of  those  runs  ended  in  completions.  All  of 
the  remaining  three  runs  were  able  to  escape  the  cul-de-sac  but  were  unable  to  complete  the  run. 
A  suspected  navigation  problem  ended  run  7  after  HMP  executed  a  k-tum  and  retreated  from  the 
blockage,  and  an  HMP  problem  ended  run  11.  Run  15  ended  for  max  backups. 

12.6  Field  Cost  Interface 

In  the  three  RIVET  runs,  the  XUV  proceeded  along  the  planned  route,  saw  that  the  path  was 
blocked,  circled  for  a  1 80°  turn  out  of  the  cul-de-sac,  and  followed  a  new  plan  back  the  trail  and 
around  the  southern  side  of  the  trees  to  the  goal.  As  with  the  AM  RIVET  runs,  the  modeled 
terrain  allowed  room  for  the  turn. 

In  the  five  field  runs,  the  XUV  encountered  the  blockage  and  backed  up  three  times  but  not  far 
enough  to  enable  the  XUV  to  see  the  unobstructed  path  to  the  goal  below  the  tree  line.  All  five 
runs  ended  with  max  backups. 


13.  Discussion  and  Analysis 


13.1  Analytical  Approach 

Analysis  of  UGV  planner  performance  is  extremely  challenging  for  several  reasons,  such  as  the 
nature  of  a  field  test,  limited  control  over  a  key  explanatory  variable,  the  lack  of  a  definitive 
performance  measure,  and  the  overall  complexity  of  the  system.  Field  experimentation  always 
introduces  greater  variability  due  to  noise  variables  not  present  in  the  sterile  surroundings  in 
most  development  testing.  This  problem  is  magnified  when  testing  UGV  planners  because 
variables  one  might  think  of  as  noise  (e.g.,  nuances  in  the  foliage,  ground  undulations)  can 
influence  perception/planning  in  ways  unanticipated  by  experimenters,  sending  the  UGV  along  a 
route  not  previously  considered. 

Beyond  local  nuances  in  the  terrain,  there  is  the  challenge  of  finding  the  exact  set  of  macro 
terrain  conditions  necessary  to  build  a  meaningful  scenario.  Even  at  FTIG,  a  relatively  large  and 
rich  area  for  terrain  variation,  finding  conditions  that  will  stress  planner  reasoning  (different  for 
each  planner)  so  that  successes  are  earned  and  failures  can  be  learned  from  is  difficult.  When 
such  a  scenario  is  seen  in  the  existing  terrain,  it  usually  requires  dressing  up  in  some  manner  with 
other  obstacles  or  a  priori  data  to  force  the  planner  to  solve  an  intended  problem  rather  than  work 
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around  the  new  challenge  by  employing  an  already  proven  planning  capability.  Previous  to  this 
study,  the  validity  of  the  planned  scenario  to  provide  the  intended  challenge  was  only  in  the  eyes 
of  an  SME. 

A  definitive  performance  measure  remains  elusive.  The  goal  should  be  to  assess  the  decision 
making  of  the  planner  as  it  progresses  toward  the  goal.  However,  evaluation  at  markers  over  the 
length  of  a  course  according  to  some  scoring  rubric  is  infeasible  because,  due  to  unanticipated 
branching  by  the  UGV,  there  is  no  way  for  the  evaluation  plan  to  consider  all  paths,  unless  in  a 
post  hoc  manner.  The  UGV  might  avoid  a  number  of  those  markers.  Time  and  distance 
measures  would  be  of  little  value.  The  objective  measure  remaining  is  only  whether  or  not  the 
UGV  overcame  the  intended  challenge  and  reached  the  goal.  A  subjective  measure  might  be 
possible  to  score  the  degree  to  which  the  UGV  behavior  on  a  scenario  was  consistent  with  SME 
expectation,  but  that  was  not  attempted  here. 

Finally,  the  complexity  of  the  technology  heightens  the  significance  of  the  three  reasons  just 
discussed.  To  begin,  there  are  a  number  of  settings  related  to  perception  and  planning  that  can 
be  tuned  to  a  specific  environment  (e.g.,  the  previously  mentioned  CellScale  in  FCI).  Even  in 
AM,  numerous  local  planning  costs  considered  make  finding  suitable  scenarios  difficult.  When 
costs  are  expanded  to  the  global  decision  space  by  FCI  and  DR,  the  SME  must  rely  on  a  notional 
understanding  of  the  planners’  intent. 

With  that  in  mind,  the  focus  of  the  analysis  was  on  the  expected  behavior  of  the  UGV  in  well- 
considered  scenarios  and  whether  or  not  the  planner  overcame  the  principal  challenge  and 
reached  the  goal.  A  collective  SME  notional  understanding  of  expected  behavior  influenced  the 
scenario  choices  and  the  planned  dressing  up  of  those  routes  in  the  global  map  or  in  the  field. 
RIVET  was  used  to  reinforce  and  augment  the  notional  understanding  and  was  valuable  in 
confirming  the  scenarios,  but  fell  short  of  providing  the  definitive  reference  distribution  for 
behavior.  While  actual  performance  is  measured  and  compared  to  RIVET  results,  analysis 
looked  beyond  RIVET  for  explanations  of  observed  UGV  behavior  when  appropriate. 

13.2  Expectations  in  RIVET 

An  initial  question  to  consider  is.  Did  the  planners  perform  in  RIVET  as  expected  in  each 
scenario?  All  three  RIVET  runs  in  each  configuration  gave  identical  results,  so  RIVET  provided 
no  variation  in  performance  within  a  set  of  experimental  conditions.  Table  9  shows  the 
percentage  of  runs  in  RIVET  that  matched  the  experimenter’s  expectations,  which  were  that  AM 
would  fail  to  reach  the  goal  and  that  DR  and  FCI  runs  would  solve  the  cul-de-sac  and  reach  the 
goal.  In  15  of  18  experimental  conditions  (scenario  4  is  not  considered),  the  planners  performed 
as  expected,  except  for  FCI  in  scenarios  1-3.  In  each  exception,  the  trail  where  the  blockage  was 
placed  was  very  constrained,  leaving  the  XUV  with  only  the  choice  of  backing  up  when  the 
block  was  encountered.  The  scenarios  were  constructed  so  that  backing  up  three  times  would  not 
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Table  9.  RIVET  runs  achieving  goal  point. 


Scenario 

Planner 

Result 

1 

2 

3 

5 

6 

7 

AM 

RIVET 

0/1  (0%) 

0/1  (0%) 

0/1  (0%) 

0/1  (0%) 

0/1  (0%) 

0/1  (0%) 

DR 

RIVET 

3/3  (100%) 

3/3  (100%) 

3/3  (100%) 

3/3  (100%) 

3/3  (100%) 

3/3  (100%) 

FCI 

3/3  (100%) 

3/3  (100%) 

3/3  (100%) 

Note:  Red  indieates  0%-24%  eompleted. 


ordinarily  enable  the  XUV  to  observe  an  alternative  route;  hence,  each  FCI  run  in  these  scenarios 
ended  in  a  stop  for  max  backups.  The  lack  of  an  HMP  interface  to  FCI  probably  contributed  to 
this  failure  to  maneuver  to  a  location  where  an  alternate  route  could  be  observed.  These  results, 
obtained  before  the  field  portion  of  the  experiment,  confirmed  the  validity  of  the  scenarios  to  the 
extent  possible  in  simulation. 

13.3  Expectations  in  the  Field 

A  second  question  to  consider  is.  Did  the  planners  perform  in  the  field  runs  as  expected  in  each 
scenario?  There  was  considerable  variation  in  the  routes  taken  and  the  outcomes  of  the  field 
runs. 

13.3.1  Reaching  the  Goal  Point 

Table  10  shows  the  outcomes  of  the  runs  in  the  field.  “Complete”  means  that  the  XUV  reached 
the  intended  goal  after  overcoming  the  obstacles  placed  in  the  original  planned  path.  Color 
codes  are  used  to  illustrate  the  percentage  of  runs  achieving  the  goal  point. 


Table  10.  Field  runs  reaching  the  goal  point. 


Note:  Green  indieates  75%-100%  mateh,  yellow  is  25%-74%  mateh,  and  red  is  0%-24%  eompleted. 
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AM  runs  went  about  as  expected.  Four  of  the  scenario  cul-de-sacs  stumped  the  XUV  totally. 
Scenario  2  was  solved  by  AM  because  the  trail  prior  to  the  block  was  not  so  dense  with  brush 
that  AM  could  not  find  an  alternate  route  around  the  block  through  the  brush  and  trees.  DR  and 
FCI  also  benefitted  from  this  location;  each  planner,  when  successful,  chose  the  identical  bypass 
of  the  cul-de-sac.  DR  reached  the  goal  point  on  64%  of  its  runs  but  completely  failed  scenario  3, 
where  it  was  expected  to  retreat  from  a  cul-del-sac  on  a  narrow  trail  on  a  steep  hill.  Two  runs 
were  e-stopped  when  HMP  backed  the  XUV  into  trees,  and  the  third  run  was  e-stopped  when  the 
XUV  went  through  the  barrels  blocking  the  trail.  FCI  was  less  effective  (48%)  than  DR  and 
completely  failed  in  scenario  7,  where  it  could  never  see  a  clear  path  after  three  backups. 

Six  runs  did  not  achieve  the  goal  point  but  did  overcome  the  blockage  and  solve  the  cul-de-sac 
problem.  After  getting  past  the  blockage,  runs  2,  7,  11,38,  46.2,  and  65  experienced  other 
mobility  problems  that  caused  an  e-stop  before  reaching  the  goal.  Figure  17  illustrates  this  point 
with  run  7  of  scenario  7.  In  this  run,  the  DR  solved  the  cul-de-sac  (upper  right)  and  moved 
toward  the  goal  point,  generating  several  new  plans  along  the  way  until  it  lost  the  NAV  solution 
and  the  run  was  e-stopped.  At  the  point  of  the  e-stop,  the  XUV  had  a  viable  plan  to  the  goal. 


Figure  17.  Run  7  where  XUV  solves  cul-de-sac  but  does  not  achieve  the  goal  point. 
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Table  1 1  shows  the  pereentage  of  rans  that  solved  the  eul-de-sac  regardless  of  whether  or  not  the 
XUV  reaehed  the  goal  point. 

Table  11.  Field  runs  solving  the  cul-de-sae. 


Scenario 


Planner 


Result 


AM 


Field 


Result 


DR 


Field 


Solved 
1 3/4  (75%) 


Solved 


3/5  (60%) 


Solved 


4/5  (80%) 


Solved 


3/4  (75%) 


Solved 


1/4  (25%) 


Solved 


5/5  (100%) 


Solved 


5/5  (100%) 


Solved 


4/5  (80%) 


Result 


FCI 


Field 


Solved 


Solved 


Solved 


Solved 


Solved 


1/3  (33%) 


3/4  (75%) 


2/3  (67%) 


2/5  (40%) 


5/5  (100%)! 


Subtotal 

6/15 

(40%) 

Total 

22/28 

(79%) 

41/68 

(60%) 

13/25 

(52%) 

Note:  Green  indieates  75%-100%  mateh,  yellow  is  25%-74%  mateh,  and  red  is  0%-24%  eompleted. 


Results  suggest  DR  and  FCI  were  very  effective  at  solving  the  cul-de-sacs,  DR  at  79%  and  FCI 
at  52%.  The  percentage  of  completed  runs  and  the  percentage  of  runs  that  “solved”  the 
cul-de-sac  are  shown  in  figure  18. 


Completed  Run  and  Solved  cul-de-sacs 


100%  100% 


AM 

DR 

FCI 

AM 

DR 

FCI 

Field 

Field 

Field 

RIVET 

RIVET 

RIVET 

□  Completed  Runs 
■  Solved  Runs 


Planner 


Figure  18.  Comparison  of  completed  runs  vs.  solved  runs. 
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13.3.2  AM  Observations 

Despite  attempts  to  design  them  otherwise,  scenarios  2  and  7  turned  out  to  be  solvable  by  AM. 
AM  runs  were  successful  in  scenario  2  because  the  shrubs  and  trees  along  the  trail  near  the 
blockage  were  not  sufficiently  dense  to  keep  the  XUV  from  finding  an  opening  after  it  backed 
away  from  the  block.  In  scenario  7,  AM  never  entered  the  cul-de-sac  on  one  run  and  was  able  to 
back  completely  out  of  the  cul-de-sac  on  a  second  run.  Except  for  these  instances,  the  AM  runs 
could  not  solve  the  scenarios. 

13.3.3  DR  Observations 

With  the  exception  of  scenario  3,  DR  runs  achieved  the  goal  on  18/24  (75%)  of  the  runs  and 
solved  21/24  (88%)  of  the  cul-de-sacs.  This  is  very  good  performance  given  the  difficulty  of  the 
scenarios.  The  steep  terrain  and  rutted  trail  on  scenario  3  prohibited  the  HMP  from  maneuvering 
the  XUV  well  at  the  point  of  the  blockage.  On  the  one  run  when  the  XUV  was  able  to  maneuver 
and  generate  new  plans,  it  failed  to  find  the  small  opening  leading  to  an  unmarked  trail  up  the 
hill  to  the  goal.  In  scenario  7,  the  failed  run  had  navigation  and  HMP  issues  discussed  in 
appendix  A.  It  could  be  argued  that  success  in  scenario  2  might  have  been,  as  with  AM,  more  a 
local  planning  achievement,  but  as  new  global  routes  pulled  it  in  that  direction  DR  is  credited. 

13.3.4  FCI  Observations 

FCI  runs  were  successful  about  half  the  time  (48%).  Scenario  7  was  singularly  difficult  for  FCI. 
In  all  five  scenario  7  runs,  the  XUV  backed  up  three  times  but  was  not  able  to  observe  an 
alternative  route;  hence,  each  of  the  FCI  runs  in  this  scenario  ended  in  a  stop  for  max  backups. 
The  lack  of  an  HMP  interface  probably  contributed  to  this  failure  to  maneuver  to  a  location 
where  an  alternate  route  could  be  observed.  The  same  factors  applied  to  failed  runs  in 
scenarios  1-3.  The  same  caveat  for  successful  scenario  2  performance  for  DR  could  be  made  for 
FCI,  but  FCI  is  credited  for  the  three  successes. 

13.4  Scenario  Difficulty 

Each  scenario  had  its  own  particular  level  of  difficulty  attributable  to  the  size  of  the  cul-de-sac, 
the  maneuver  room  available,  the  density  of  the  trees  and  shrubs,  etc.  When  data  was  used  from 
only  the  DR  and  FCI  runs,  and  based  on  the  percentage  of  runs  completed,  scenarios  3  and  7 
were  the  most  difficult  for  the  planners  to  solve  (see  figure  19).  The  blockage  in  scenario  3  was 
on  a  narrow  trail  on  a  steep  hill — a  very  difficult  mobility  problem  for  the  XUV.  Scenario  7 
difficulty  was  caused  by  the  depth  of  the  cul-de-sac  and  the  lack  of  maneuver  room  near  the 
blockage.  The  XUV  was  often  not  able  to  turn  around  or  back  up  far  enough  to  exit  the  cul-de- 
sac. 
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scenario  difficulty  (DR  and  FCI  only) 


28.6% 

3  5 

Scenario  # 


Figure  19.  Scenario  difficulty  (percent  goal  achieved  DR  and  FCI  only). 


14.  Excursion  Scenario  1.1 


14.1  Situation 

Scenario  1 . 1  was  the  same  as  scenario  1  except  that  the  NBC  area  was  removed  in  order  to  see  if 
the  DR  would  plan  to  use  the  low-cost  main  tank  trail  to  the  east  as  an  alternate  route  (figure  20). 
Added  as  an  excursion  during  the  field  experiment,  scenario  1.1  was  not  run  in  RIVET. 


Figure  20.  Scenario  1.1  description. 
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14.2  Dynamic  Replanner 

On  neither  run  did  the  DR  plan  a  path  that  utilized  the  main  tank  trail  to  the  east.  Run  1 16 
(figure  21)  was  typical  of  the  DR  runs  31.3  and  42,  where  the  XUV  saw  the  blockage,  returned 
to  the  opening  on  the  right,  discovered  the  unmarked  trail  to  the  left,  and  proceeded  slowly  down 
the  unmarked  trail  under  HMP  control.  The  DR  generated  plans  that  directed  the  XUV  back  to 
the  original  trail,  beyond  the  blockage.  However,  the  XUV  was  unable  to  find  a  clear  path 
through  the  brush  and  trees,  and  eventually  the  DR  gave  a  plan  that  followed  the  unmarked  trail 
to  the  goal.  On  run  1 14  (figure  21),  the  XUV  was  e-stopped  when  the  HMP  backed  it  into  an 
unknown  area  of  high  brush  near  the  blockage. 


Figure  21 .  Scenario  1 . 1  runs  with  no  NBC  blockage. 


15.  Excursion  Scenario  6.1 


15.1  Situation 

Scenario  6.1  was  exactly  the  same  as  scenario  6  with  one  exception  (figure  22).  The  first 
alternative  path,  the  trail  out  of  the  finger  area  north  of  the  original  path,  was  not  blocked.  This 
constituted  a  scenario  not  quite  as  challenging  as  scenario  6.  This  scenario  was  an  ad  hoc 
addition  to  the  experiment  and  was  not  run  in  RIVET. 
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Figure  22.  Scenario  6.1  description. 

15.2  Expectations 

The  expectations  were  that  (1)  AM  would  encounter  the  blockage  on  the  original  path  and 
possibly  find  the  opening  to  the  trail  to  the  north,  and  (2)  upon  encountering  the  initial  blockage, 
the  DR  and  FCI  would  generate  new  plans  to  the  nearby  alternate  trail  to  the  north. 

15.3  Results 

Table  12.  Scenario  6.1  results. 


Scenario 

# 

Planner 

Source 

Run 

Rans 

Dist  (m) 

Total  time 
(min) 

eng_off 

(min) 

Time  HMP 
(min) 

ET  (min) 

avg  spd 
kph 

Backups 

Outcome 

6.1 

AM  only 

Field 

109 

1 

742.09 

6.57 

6.57 

6.78 

1 

circles 

6.1 

AM  only 

Field 

114 

483.67 

5.09 

5.09 

5.71 

4 

circles 

6.1 

DR 

Field 

107 

8 

232.81 

1.58 

0.12 

1.45 

1.46 

9.60 

Complete 

6.1 

DR 

Field 

112 

7 

247.39 

1.66 

0.03 

0.61 

1.62 

9.15 

Complete 

6.1 

FCI 

Field 

108 

1 

254.90 

2.04 

0.29 

1.75 

8.74 

Complete 

6.1 

FCI 

Field 

113 

1 

233.31 

1.34 

0.02 

1.32 

10.60 

Complete 

15.4  Autonomous  Mobility 

In  the  two  field  runs,  the  XUV  repeatedly  circled  in  the  cul-de-sac  and  never  came  close  enough 
to  the  unblocked  trail  in  the  north  to  see  the  opening.  After  circling  three  or  more  times,  the  run 
was  e-stopped. 

15.5  Dynamic  Replanner 

In  the  two  field  runs,  the  XUV  sensed  the  blockage  in  the  original  path  and  quickly  generated 
and  executed  a  new  global  path  through  the  alternate  trail  to  the  north.  These  runs  spent  37% 
and  92%  of  the  time  in  HMP  at  the  cul-de-sac  before  generating  the  new  global  plan  that  worked. 
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15.6  Field  Cost  Interface 


In  the  two  field  nins,  the  XUV  sensed  the  bloekage  in  the  original  path  and  quickly  generated 
and  executed  a  new  path  through  the  alternate  trail  to  the  north.  This  occurred  quickly  and 
seamlessly  on  both  runs. 


16.  Excursion  Scenario  6.2 


16.1  Situation 

Scenario  6.2  was  exactly  the  same  as  scenario  6.  This  scenario  was  an  ad  hoc  addition  to  the 
experiment  and  was  not  run  in  RIVET.  The  weight  for  local/global  costs  in  the  FCI  planner 
calculations  was  varied  in  two  runs  using  the  FCI  planner  only. 

16.2  Expectations 

Observing  the  performance  of  FCI  in  scenario  6  and  noting  the  number  of  times  the  XUV  circled 
the  cul-de-sac  before  generating  a  path  out  of  the  cul-de-sac,  the  experimenters  decided  to  run 
the  same  scenario  but  with  a  different  ratio  of  between  the  local  and  global  planner  weights  for 
FCI.  The  variable  CellScale  was  reduced  from  0.2  to  0.1  for  the  excursion,  giving  more  weight 
to  the  local  planner,  which  was  less  forgiving  of  the  obvious  brush  in  the  immediate  path  and 
caused  the  XUV  to  more  quickly  generate  a  path  out  of  the  cul-de-sac. 

16.3  Results 

Table  13.  Scenario  6.2  results. 


Scenario 

# 

Ranner 

Source 

Run 

Rans 

Dist  (m) 

Total  time 
(min) 

eng_off 

(min) 

Time  HMP 
(min) 

ET  (min) 

avg  spd 
kph 

Backups 

Outcome 

6.2 

FCI 

Field 

110 

1 

860.47 

7.90 

7.90 

6.53 

1 

Complete 

6.2 

FCI 

Field 

111 

1 

446.21 

4.31 

0.47 

3.84 

6.97 

2 

E-stop 

16.4  Field  Cost  Interface 

As  expected,  on  both  runs,  the  XUV  exited  the  cul-de-sac  area  after  circling  twice  near  the 
blockages.  One  run  was  completed,  and  the  second  run  was  e-stopped  when  the  XUV  ran  into 
undetected  and  unmarked  water  after  exiting  the  cul-de-sac. 
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17.  Effects  of  Varying  CellScale 


FCI  runs  in  scenarios  5  and  6  successfully  solved  the  cul-de-sac  after  several  “circles”  of  the 
cul-de-sac  area.  Several  looks  at  the  terrain  were  required  in  order  to  cause  the  perception  system 
to  recognize  the  local  blockages  and  increase  the  local  cost,  enabling  the  FCI  planner  to  then  select 
a  plan  that  took  the  XUV  out  of  the  cul-de-sac  area.  FCI  runs  circled  the  area  an  average  of  4.2 
times  when  the  CellScale  was  set  to  0.2.  Data  from  these  runs  are  found  in  table  14. 


Table  14.  Number  of  circles  for  FCI 
runs  in  scenarios  5  and  6. 


Run 

No. 

No.  of 
Circles 

CellScale 

Source 

21.1 

3 

0.2 

Field 

23 

3 

0.2 

Field 

25.2 

8 

0.2 

Field 

28 

3 

0.2 

Field 

76 

5 

0.2 

Field 

79.1 

2 

0.2 

Field 

82 

4 

0.2 

Field 

85 

8 

0.2 

Field 

88 

2 

0.2 

Field 

Average  No.  of  Circles  =  4.2 

While  not  conclusive,  this  is  evidence  that  CellScale  is  influential  in  the  performance  of  the  FCI 
planner  and  should  be  investigated  using  different  levels  and  various  terrain  situations. 


18.  Similarities  and  Differences  Between  RIVET  and  Field  Runs 


18.1  RIVET 

RIVET  was  an  invaluable  tool  in  preparing  for  the  assessment,  ft  provided  a  test  bed  for  the 
integrated  planners,  which  would  not  have  been  available  otherwise,  ft  has  been  used  to 
replicate  and  test  bugs  and  other  issues  that  are  only  present  when  running  all  the  processes  and 
sensors  present  in  the  XUV.  It  also  provides  a  more  controlled  environment  with  generous 
display  options  in  which  multiple  people  can  work  together  when  trying  to  troubleshoot  the 
integrated  system.  Lastly,  it  proved  to  be  a  valuable  training  tool  that  reduces  the  learning  curve 
for  operating  the  XUV  and  the  different  planners.  In  many  of  the  scenarios,  RIVET’ s 
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representation  was  so  good  that  the  behavior  of  the  XUV  in  the  field  matched  the  simulation 
even  in  small  nuances  and  exercised  some  bugs  that  were  hard  to  reproduce.  Still,  there  are  a 
number  of  areas  in  which  there  is  room  for  improvement. 

18.2  Matched  Runs 

As  illustrated  in  figure  23,  in  some  instances,  RIVET  and  field  runs  matched  very  closely. 


RIVET  Run  260  and  Field  Run  89 
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4480290  ■ 

4480240  - 

4480190  ■ 
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362850  382950  363050  363150  363250  363350 


Figure  23.  Example  of  closely  matching  RIVET  and  field  runs. 


Table  15  shows  the  extent  to  which  the  XUV  tracks  in  field  runs  matched  the  XUV  tracks  in 
RIVET  runs.  Overall,  about  half  (48%)  of  the  field  runs  closely  resembled  a  RIVET  run. 

Table  15.  Correlations  between  RIVET  and  field  results. 
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18.3  Lack  ofVariability  in  RIVET 

For  all  scenarios  and  each  planner,  the  three  RIVET  runs  produced  the  same  end  result  and 
almost  identical  tracks.  There  was  no  variability  between  runs.  There  was  at  least  one  field  run 
for  each  experimental  condition  that  was  very  similar  to  corresponding  RIVET  runs;  however, 
for  any  given  set  of  conditions,  the  lack  of  variability  in  RIVET  is  a  significant  limitation.  The 
plots  in  figure  24  show  the  repeatability  of  RIVET  runs.  Ideally,  in  subsequent  studies,  there 
would  be  the  ability  to  introduce  degrees  of  variability  into  the  terrain  to  elicit  a  distribution  of 
behaviors.  That  distribution  would  then  serve  as  a  reference  for  observed  performance 
evaluations.  There  is  no  guarantee  that  the  specific  lay  down  of  simulated  terrain  for  each 
scenario  in  this  study  was  in  any  way  the  best  lay  down.  Further,  the  capability  to  easily 
introduce  variable  terrain  would  likely  enhance  the  developers’  opportunities  to  experience 
interesting  planner  behaviors  in  the  lab,  where  the  feedback  loop  in  software  development  is 
more  efficient. 


Scenario  4  -  new  barrel  locations 


ei*  ye 


■ ,  fl  * 


Pifctdwt 

1  iJ  El  ^  hhic^dtt  (Ul«4  | 

.  * 

Figure  24.  Examples  of  lack  of  variability  in  RIVET. 
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18.4  E-stops 

There  were  no  e-stops  in  RIVET.  All  RIVET  runs  ended  in  a  completed  run  or  in  max  backups. 
In  the  current  implementation,  HMP  was  allowed  to  back  into  unknown  areas,  resulting  in  some 
e-stops  in  the  field  but  with  no  consequences  in  RIVET.  An  effort  might  be  considered  to  elicit 
from  program  participants  the  anatomy  of  an  e-stop  and  include  that  run  outcome  in  RIVET. 

18.5  HMP 

During  the  DR  runs,  the  XUV  spent  considerably  less  time  in  HMP  in  RIVET  vs.  field  runs  (see 
figure  25).  HMP  works  flawlessly  in  RIVET  and  achieves  much  higher  speeds  than  in  the  field 
runs. 


Figure  25.  Time  spent  in  HMP. 


18.6  Backups 

There  appears  to  be  no  significant  difference  in  the  number  of  backups  the  XUV  performed  in 
RIVET  vs.  field  runs.  Reasons  for  the  two  spikes  in  the  number  of  backups  in  figure  26  are  not 
clear.  In  RIVET,  scenario  3,  FCI  run  286  had  27  backups,  run  287  had  3  backups,  and  run  288 
had  13  backups.  In  RIVET,  scenario  6,  AM  run  261  had  5  backups,  run  262  had  6  backups,  and 
run  263  had  12  backups.  An  explanation  would  require  additional  investigation. 
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Figure  26.  Number  of  XUV  backups. 

18.7  Performance  of  FCI  in  Scenarios  5  and  6 

As  discussed  previously,  FCI  runs  in  scenario  5  and  6,  which  successfully  solved  the  cul-de-sae 
or  were  halted  after  numerous  eircles  of  the  cul-de-sae  area,  circled  the  area  an  average  of 
4.1  times  when  CellSeale  was  set  to  0.2.  Corresponding  runs  in  RIVET  are  shown  in  table  16. 


Table  16.  Number  of  FCI  circles  in 

scenarios  5  and  6  using  RIVET. 


Run 

No. 

No.  of 
Circles 

CellSeale 

Source 

12 

2 

0.2 

RIVET 

13 

1 

0.2 

RIVET 

215 

1 

0.2 

RIVET 

251 

1 

0.2 

RIVET 

254 

1 

0.2 

RIVET 

255 

1 

0.2 

RIVET 

Average  No.  of  Circles  = 

1.2 

Although  FCI  solved  scenarios  5  and  6  eonsistently  in  the  field,  this  data  shows  a  signifieant 
difference  in  the  “effieiency”  of  FCI  in  those  scenarios  relative  to  the  eorresponding  RIVET 
runs. 
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18.8  Reasons  for  End  of  Mission 

Table  17  summarizes  the  reasons  that  an  early  end  of  mission  was  ealled  on  experimental  runs 
that  were  not  completed. 


Table  17.  Reasons  for  early  end  of  mission  calls. 


Source 

Planner 

Reason 

Count 

Field 

AM  only 

E-stop 

3 

Field 

AM  only 

Max  backups 

3 

Field 

AM  only 

Stuck 

1 

Field 

AM  only 

Three  circles 

5 

Field 

DR 

E-stop 

3 

Field 

DR 

E-stop  HMP 

5 

Field 

DR 

Max  backups 

1 

Field 

DR 

Stuck 

1 

Field 

FCI 

E-stop 

4 

Field 

FCI 

Max  backups 

9 

Field 

FCI 

Three  circles 

1 

RIVET 

AM  only 

Max  backups 

18 

RIVET 

FCI 

Max  backups 

12 

Five  runs  were  ended  prematurely  for  DR  runs  when  HMP  backed  the  XUV  into  unknown  areas 
that  were  brush  or  trees,  causing  the  e-stop.  This  is  a  known  condition  for  HMP  and  has  been 
addressed  for  future  implementation. 

FCI  runs  were  ended  nine  times  for  max  backups.  Although  plans  generated  by  FCI  take  into 
account  the  kinematics  of  the  XUV,  this  precludes  the  kinds  of  tight  maneuvers  possible  under 
HMP  control.  It  appears  that  integration  of  HMP  into  the  FCI  planner  would  significantly 
improve  its  performance. 


19.  Implications  for  Operator  Performance 


This  field  assessment  did  not  assess  operator  performance  or  the  need  for  operator  interventions. 
However,  there  are  potential  implications  for  operator  performance  resulting  from  these  kinds  of 
planning  algorithms. 
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As  UGV  intelligence  becomes  greater  and  increasingly  autonomous  tactical  behaviors  are 
implemented,  the  role  of  the  human  operator  will  change.  Planning  algorithms,  such  as  those 
described  and  assessed  in  this  report,  will  allow  the  robot  to  execute  plans  and  reach  the  end  goal 
with  less  human  intervention.  The  robot  will  have  the  capability  to  achieve  the  end  goal  with 
flexibility  as  to  the  specific  route  taken.  The  UGV  will  be  able  to  pick  and  choose  its  route  even 
as  the  actual  terrain  presents  obstacles  and  challenges.  The  operator  will  have  fewer  direct 
control  tasks  and  be  called  for  fewer  interventions,  thus  reducing  operator  workload. 

It  is  important,  though,  to  consider  other  potential  implications  for  operator  performance.  While 
there  may  be  less  direct  control  tasks  required,  the  operator  will  still  be  required  to  periodically 
monitor  and  supervise  the  UGV  progress.  For  some  activities,  such  as  the  cul-de-sac  problem 
(e.g.,  see  scenario  6),  the  operator  could  see,  at  first,  the  UGV  moving  toward  the  goal  and  then, 
some  time  later,  see  the  same  UGV  heading  in  the  exact  opposite  direction.  For  the  operator,  this 
behavior  may  be  inexplicable. 

A  major  need  will  be  some  way  for  the  operator  to  understand  UGV  actions  and  determine  if  its 
behavior  is  appropriate  or  unreliable.  What  information  is  needed  and  how  it  is  conveyed  to  the 
operator  is  an  open  question  and  should  be  considered  as  the  technology  develops.  Considering 
and  addressing  the  implications  for  operator  performance  will  enable  more  successful  interaction 
between  humans  and  unmanned  intelligent  systems  so  that  such  systems  can  be  used  to  their  full 
capability  in  an  operational  environment. 


20.  Conclusions 


Both  DR  and  FCI  are  significant  improvements  over  AM  in  improving  the  ability  of  the  XUV  to 
solve  complex  terrain  blockages  and  to  plan  and  execute  alternate  routes  to  a  goal  point.  For  a 
quick  summary,  the  results  are  again  presented  in  tables  18  and  19.  Overcoming  such  challenges 
represents  a  major  improvement  in  maneuvering  in  complex  terrain  and  is  an  enabler  for  tactical 
behaviors. 

It  is  not  fair  to  compare  DR  and  FCI  results  because  of  the  effects  of  HMP  that  are  integrated 
with  DR  but  not  FCI.  HMP  seemed  to  enhance  the  capability  of  DR,  allowing  some  intricate 
maneuvers  to  follow  a  new  global  route.  At  the  same  time,  HMP  was  the  direct  cause  of  five 
e-stops  because  of  a  known  limitation  allowing  an  XUV  back  up  into  an  area  unknown  to  its 
map.  FCI  was  stopped  in  nine  runs  because  of  max  backups,  meaning  it  could  not  maneuver  out 
of  the  tight  constraints  around  a  blockage,  a  capability  provided  by  HMP. 

Changes  in  FCI  parameters  that  affect  the  weighting  between  local  and  global  plans  significantly 
affect  the  efficiency  of  FCI  in  solving  terrain  blockages  (see  excursion  6.2). 
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Table  18.  Goal  achieved/runs  attempted. 


Goal  Achieved 

Scenario 

RIVET 

Field 

AM 

DR 

FCI 

AM 

DR 

FCI 

1 

0/3 

3/3 

0/3 

0/1 

3/5 

1/3 

2 

0/3 

3/3 

0/3 

3/4 

3/4 

2/4 

3 

0/3 

3/3 

0/3 

0/2 

0/4 

2/3 

4 

0/3 

3/3 

0/3 

— 

— 

— 

5 

0/3 

3/3 

3/3 

0/2 

5/5 

2/5 

6 

0/3 

3/3 

3/3 

0/1 

5/5 

5/5 

7 

0/3 

3/3 

3/3 

2/5 

2/5 

0/5 

Table  19.  Challenges  solved/challenges  attempted. 


Challenges  Solved 

Scenario 

RIVET 

Field 

AM 

DR 

FCI 

AM 

DR 

FCI 

1 

0/3 

3/3 

0/3 

0/1 

4/5 

1/3 

2 

0/3 

3/3 

0/3 

3/4 

3/4 

3/4 

3 

0/3 

3/3 

0/3 

0/2 

1/4 

2/3 

4 

0/3 

3/3 

0/3 

— 

— 

— 

5 

0/3 

3/3 

3/3 

0/2 

5/5 

2/5 

6 

0/3 

3/3 

3/3 

0/1 

5/5 

5/5 

7 

0/3 

3/3 

3/3 

3/5 

4/5 

0/5 

Lack  of  variability  of  performance  in  RIVET  limits  its  usefulness  and  its  validity  with  respect  to 
field  runs  toward  building  a  complete  reference  distribution  of  expected  behaviors.  Still,  48%  of 
the  field  runs  are  very  similar  to  corresponding  RIVET  runs.  Even  without  variability 
introduced,  RIVET  is  invaluable  in  confirming/augmenting  SME  notional  understanding  of 
planner  behavior  in  the  scenario  planning  stages. 

DR  and  FCI  greatly  reduce  the  need  for  operator  intervention  to  assist  the  XUV  in  overcoming 
an  unexpected  terrain  blockage.  On  DR  and  FCI  runs,  no  operator  intervention  was  required. 

No  measure  of  operator  workload  reduction  was  collected. 

For  DR  and  FCI,  locally  sensed  data  was  used  to  update  or  replace  a  priori  data  in  both  the  local 
and  global  maps.  Both  planners  were  able  to  explore  alternate  routes  through  areas  where 
a  priori  data  did  not  exist. 

The  one  instance  of  the  use  of  false  a  priori  data  (scenario  3)  was  inconclusive.  One  DR  run 
attempted  to  use  the  false  a  priori  trail  in  its  plan,  but  the  locally  sensed  data  showed  the  area  as 
impassible  and  alternate  routes  were  generated  (see  field  run  65). 
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The  scenarios  used  in  this  technology  assessment  to  evaluate  the  performance  of  the  planning 
algorithms  offered  significant  challenges  to  the  XUV.  The  results  show  that  beneficial 
improvements  have  been  made  over  the  previous  ability  of  the  vehicle  to  take  advantage  of 
sensed  data  for  recognizing  a  blockage  and  planning  alternative  paths  to  the  goal.  It  is  hoped  that 
as  improvements  are  made  in  sensor  range  and  resolution  (e.g.,  midrange  sensing)  and  these 
capabilities  are  integrated  into  the  planning  hierarchy,  the  UGV  will  be  able  to  maneuver  more 
effectively.  This  should  reduce  the  number  of  pitfalls  encountered  by  the  vehicle,  such  as 
entering  a  cul-de-sac  or  a  blocked  road.  The  value  of  continued  research  in  sophisticated  path 
planning  algorithms  is  that  the  vehicle  will,  at  some  point,  be  able  to  learn  from  its  environment 
and  self-adjust  the  parameters  required  to  accommodate  scenarios  that  differ  in  terrain  type  and 
complexity.  Although  significant  progress  has  been  made  in  enabling  an  unmanned  autonomous 
vehicle  to  use  local  and  global  planning  to  overcome  a  number  of  difficult  scenarios,  a 
considerable  amount  of  work  will  be  required  to  realize  a  system  that  can  perform  in  a  robust 
manner  when  faced  with  the  challenges  found  in  the  various  unstructured,  dynamic  environments 
that  a  UGV  will  undoubtedly  encounter. 
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Intentionally  left  blank. 
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Appendix  A.  Graphics  and  Narrative  of  All  Runs 


A.1  Scenario  1 

A.1.1  Autonomous  Mobility  Only 

The  three  robotie  interactive  visualization  and  exploitation  technology  (RIVET)  runs  with  the 
Autonomous  Mobility  (AM)-only  Planner  resulted  in  identical  behaviors.  The  experimental 
unmanned  vehicle  (XUV)  moved  down  the  trail,  saw  the  blockage,  and  backed  up  three  times; 
the  run  was  then  halted  (figure  A-1). 


Figure  A-1.  Run  238-AM-RIVET. 


One  field  run  (run  33)  with  AM  only  was  conducted  and  similarly  ended  after  max  backups.  AM 
only  was  unable  to  find  an  alternate  route  (figure  A-2). 


Figure  A-2.  Run  33-AM-Field. 
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A.  1.2  Dynamic  Replanner 


The  three  RIVET  runs  with  the  Dynamic  Replanner  (DR)  resulted  in  identical  behaviors.  The 
XUV  moved  down  the  trail  and  saw  the  blockage.  The  High-Maneuverability  Planner  (HMP) 
then  maneuvered  the  XUV  back  to  the  point  where  an  opening  existed  to  the  southeast.  The 
XUV  then  drove  through  the  opening  and  discovered  an  unmarked  trail  to  its  left  and  parallel  to 
the  original  plan,  and  followed  that  trail  to  the  goal.  In  the  RIVET  runs,  HMP  was  invoked 
99.7%  of  the  time.  All  three  RIVET  runs  achieved  the  goal.  The  orange  triangles  appearing  in 
the  figures  of  DR  runs  represent  the  location  of  the  XUV  when  a  new  global  plan  to  the  goal  is 
accepted  by  the  XUV  (figure  A-3).  Hence,  many  new  plans  are  observed  in  cul-de-sac 
situations,  and  fewer  new  plans  are  observed  in  less  congested  environments  where  the  XUV  is 
moving  easily  toward  the  goal. 


Figure  A-3 .  Run  23 1  -DR-RIVET. 


Three  of  the  five  field  runs  achieved  the  goal.  On  runs  31.3  and  42  (figures  A-4  and  A-5),  the 
XUV  saw  the  blockage,  returned  to  the  opening  on  the  southeast,  discovered  the  unmarked  trail 
to  its  left,  and  proceeded  slowly  down  the  unmarked  trail  under  HMP  control.  The  DR 
continuously  generated  plans  that  directed  the  XUV  back  to  the  original  trail,  beyond  the 
blockage.  However,  the  XUV  was  unable  to  find  a  clear  path  through  the  brush  and  trees,  and 
eventually  the  DR  gave  a  plan  that  followed  the  unmarked  trail  to  the  goal. 
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Figure  A-4.  Run  3 1 . 1-DR-Field. 


Figure  A-5.  Run  42-DR-Field. 
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On  run  35  (figure  A-6),  the  XUV  followed  a  similar  path  but  found  a  clear  path  back  to  the 
original  trail  and  completed  the  run  this  way. 


Figure  A-6.  Run  35-DR-Field. 


Runs  38  and  43  (figures  A-7  and  A-8)  ended  in  e-stops  when  the  HMP  backed  the  XUV  into  the 
dense  brush,  which  was  an  unknown  area. 


Figure  A-7.  Run  38-DR-Field 
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Figure  A-8.  Run  43-DR-Field. 

Two  of  the  five  field  runs  closely  resembled  the  RIVET  runs. 

A.1.3  Field  Cost  Interface 

None  of  the  three  RIVET  runs  achieved  the  goal  point.  All  Field  Cost  Interface  (FCI)  runs  in 
RIVET  saw  the  blockage  and  backed  up  three  times;  the  runs  were  then  halted.  Run  226  (figure 
A-9)  is  typical. 


Figure  A-9.  Run  226-FCI-RIVET. 
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Field  runs  34  and  39  (figures  A-10  and  A-1 1)  were  very  similar  to  RIVET  runs,  stopping  after 
three  baekups. 


Figure  A-10.  Run  34-FCI-Field. 


Figure  A-1 1 .  Run  39-FCl-Field. 
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On  run  32  (figure  A- 12),  the  XUV  baeked  up  three  consecutive  times  without  moving  forward. 
This  combined  60-m  backup  enabled  it  to  discover  the  opening  to  the  southeast  and  the 
unmarked  trail  paralleling  the  original  path  to  the  goal.  The  XUV  proceeded  down  the  unmarked 
trail  to  the  goal. 


Figure  A-12.  Run  32-FCI-Field. 


A.2  Scenario  2 

A.2.1  Autonomous  Mobility  Only 

The  three  RIVET  runs  with  the  AM-only  planner  resulted  in  identical  behaviors  (figure  A- 13). 
The  XUV  moved  down  the  trail,  saw  the  blockage,  and  backed  up  three  times.  The  runs  were 
ended  after  three  consecutive  backups.  AM  only  was  unable  to  find  an  alternate  route. 


Figure  A- 1 3 .  Run  24 1  -AM-RIVET. 
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Three  of  the  four  AM-only  field  runs  (47,  51,  and  52)  were  able  to  successfully,  but 
unexpectedly,  find  an  alternate  route  to  the  east  of  the  blockage  (figures  A-14-A-16).  This  was 
probably  the  result  of  sparse  enough  brush  in  the  area  for  AM  to  find  a  local  path  and  move  the 
XUV  through  to  the  goal.  With  each  successful  run  through  the  brush,  the  next  run  became 
easier  for  the  planner  to  find  a  path  through  the  brush. 


Figure  A-14.  Run  47-AM-Field. 


Figure  A-15.  Run  5 1-AM-Field. 
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Figure  A-16.  Run  52-AM-Field. 

Run  57  (figure  A- 17)  ended  in  an  e-stop  when  the  XUV  backed  too  close  to  a  high-mobility 
multipurpose  wheeled  vehicle  (HMMWV)  used  for  blockage. 


Figure  A- 17.  Run  57-AM-Field. 

None  of  the  field  runs  were  similar  to  the  RIVET  runs. 

A.2.2  Dynamic  Replanner 

The  three  RIVET  runs  with  the  DR  resulted  in  identical  behaviors  (figure  A- 18).  The  XUV 
moved  down  the  trail  and  saw  the  blockage.  HMP  then  maneuvered  the  XUV  back  to  the  point 
where  an  opening  existed  to  a  trail  to  the  east.  The  XUV  then  followed  the  trail  to  the  goal.  In 
the  RIVET  runs,  HMP  was  invoked  76%-86%  of  the  time. 
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Figure  A- 1 8.  Run  206-DR-RIVET. 

Three  of  the  four  field  runs  achieved  the  goal.  On  runs  48,  49,  and  53  (figures  A-19-A-21),  the 
XUV  saw  the  blockage,  generated  new  plans  to  the  east  of  the  blockage,  and  found  its  way  to  the 
goal.  No  plans  were  generated  on  these  runs  that  would  have  utilized  the  trail  to  the  west, 
leading  to  speculation  that  the  alternate  trail  was  not  in  the  a  priori  data  as  it  was  in  RIVET.  As 
with  AM  only,  the  more  trips  through  the  shrubs,  the  wider  and  easier  the  path  became. 


Figure  A-19.  Run  48-DR-Field. 
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Figure  A-20.  Run  49-DR-Field. 


F igure  A-2 1 .  Run  5  3  -DR-F ield. 
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On  run  56  (figure  A-22),  the  XUV  was  e-stopped  after  two  backups  because  it  backed  into  dense 
shrubs. 


Figure  A-22.  Run  56-DR-Field. 

None  of  the  field  runs  resembled  the  RIVET  runs. 

A.2.3  Field  Cost  Interface 

None  of  the  three  RIVET  runs  achieved  the  goal  point;  all  RIVET  runs  ended  after  three  backups 
shortly  after  encountering  the  blockage  (figure  A-23). 


Figure  A-23.  Run  220-FCI-RIVET. 


60 


Runs  50  and  54  (figures  A-24  and  A-25)  achieved  the  goal.  These  runs  were  similar  to  the 
successful  DR  and  AM-only  runs  because  the  XUV  saw  the  blockage,  moved  off  the  trail  to  the 
east  of  the  blockage  through  high  shrubs,  and  found  its  way  to  the  goal.  As  with  previous  runs, 
the  more  trips  through  the  shrubs,  the  wider  and  easier  the  path  became. 


Figure  A-24.  Run  50-FCI-Field. 


Figure  A-25.  Run  54-FCI-Field. 
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Run  46.2  (figure  A-26)  was  e-stopped  because  of  max  backups,  and  run  55. 1  (figure  A-27)  was 
e-stopped  because  it  backed  too  close  to  the  HMMWV  used  in  the  blockage. 


Figure  A-26.  Run  46.2-FCI-Field. 


Figure  A-27.  Run  55.1 -FCI-Field. 


One  field  run  was  similar  to  the  three  unsuccessful  RIVET  runs. 
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A.3  Scenario  3 


A.3.1  Autonomous  Mobility  Only 

The  three  RIVET  runs  with  the  AM-only  planner  resulted  in  identical  behaviors  (figure  A-28). 
The  XUV  moved  up  the  trail,  saw  the  blockage,  and  backed  up  three  times.  The  runs  were  ended 
after  three  consecutive  backups.  AM  only  was  unable  to  find  an  alternate  route. 


Figure  A-28.  Run  293-AM-RIVET. 

The  two  field  runs  with  the  AM-only  planner  (runs  63  and  64)  resulted  in  behavior  identical  to 
the  RIVET  runs  (figures  A-29  and  A-30).  The  XUV  moved  up  the  trail,  saw  the  blockage,  and 
backed  up  three  times.  The  trail  at  the  point  of  the  blockage  was  steep  and  deeply  rutted,  making 
any  maneuver  difficult.  The  runs  were  ended  after  three  consecutive  backups.  AM  only  was 
unable  to  find  an  alternate  route. 


Figure  A-29.  Run  63-AM-Field. 
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Figure  A-30.  Run  64-AM-Field. 


A.3.2  Dynamic  Replanner 

The  three  RIVET  runs  with  the  DR  resulted  in  identical  behaviors.  The  XUV  moved  up  the  trail 
and  saw  the  blockage.  HMP  then  maneuvered  the  XUV  back  to  the  point  where  an  opening 
existed  to  the  west.  DR  then  generated  successive  plans  to  move  the  XUV  to  the  goal  through  an 
unmarked  trail.  The  a  priori  road  to  the  east  was  not  explored.  In  the  RIVET  runs,  HMP  was 
invoked  57%-77%  of  the  time. 

None  of  the  four  field  runs  achieved  the  goal.  On  runs  62  and  68  (figures  A-3 1  and  A-32),  the 
XUV  saw  the  blockage  and  generated  new  plans  back  to  the  point  where  an  opening  existed  to 
the  west.  However,  HMP  was  unable  to  maneuver  the  XUV  back  down  the  steep  trail,  and  both 
runs  were  e-stopped  when  HMP  backed  into  high  brush. 


Figure  A-3 1.  Run  62-DR-Field. 
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Figure  A-32.  Run  68-DR-Field 


During  run  72  (figure  A-33),  the  XUV  went  through  the  barrels  at  the  blockage  and  was 
e-stopped. 


Figure  A-33.  Run  72-DR-Field. 


On  run  65  (figure  A-34),  the  XUV  maneuvered  back  to  the  base  of  the  hill,  explored  the  “false” 
a  priori  trail  to  the  east,  found  it  impassable,  and  generated  plans  to  the  west  of  the  original  plan 
toward  the  unmarked  trail  to  the  goal.  However,  the  XUV  failed  to  find  the  small  opening  in  the 
brush  for  the  unmarked  trail  and  was  e-stopped  short  of  the  goal.  In  the  field  runs,  HMP  was 
invoked  25%-85%  of  the  time. 
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Figure  A-34.  Run  65-DR-Field. 

None  of  the  field  runs  resembled  the  RIVET  runs. 

A.3.3  Field  Cost  Interface 

The  three  RIVET  runs  with  the  FCI  planner  resulted  in  identical  behaviors  (figure  A-35).  The 
XUV  moved  up  the  trail,  saw  the  blockage,  and  backed  up  three  times  until  the  run  was  halted 
for  max  backups. 


Figure  A-35.  Run  286-FCI-RIVET. 
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Two  of  the  three  FCI  field  runs  achieved  the  goal.  On  runs  61  and  69  (figures  A-36  and  A-37), 
the  XUV  saw  the  blockage,  backed  to  the  base  of  the  hill,  and  immediately  explored  the  area  to 
the  west  of  the  original  plan,  discovering  the  unmarked  trail  to  the  goal. 


Figure  A-36.  Run  61-FCI-Field. 


Figure  A-37.  Run  69-FCI-Field. 
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In  ran  66  (figure  A-38),  the  XUV  moved  up  the  trail,  saw  the  blockage,  and  backed  up  six  times 
until  the  ran  was  halted  for  max  backups. 


Figure  A-38.  Run  66-FCI-Field. 


A.4  Scenario  4 

A.4.1  Autonomous  Mobility  Only 

Scenario  4  was  only  ran  in  RIVET.  The  three  RIVET  runs  with  the  AM-only  planner  resulted  in 
identical  behaviors  (figure  A-39).  The  XUV  moved  up  the  trail,  saw  the  blockage,  and  backed 
up  three  times.  The  runs  were  ended  after  three  consecutive  backups.  AM  only  was  unable  to 
find  an  alternate  route. 


Figure  A-39.  Run  283-AM-RIVET. 
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A.4.2  Dynamic  Replanner 


The  three  RIVET  runs  with  the  DR  resulted  in  identical  behaviors  (figure  A-40).  The  XUV 
moved  up  the  trail  and  saw  the  blockage.  HMP  then  maneuvered  the  XUV  back  to  the  point 
where  an  opening  existed  to  the  west.  DR  then  generated  successive  plans  to  move  the  XUV  to 
the  goal  through  an  unmarked  trail. 


Figure  A-40.  Run  280-DR-RIVET. 

A.4.3  Field  Cost  Interface 

The  three  RIVET  runs  with  the  FCI  planner  resulted  in  identical  behaviors  (figure  A-41).  The 
XUV  moved  up  the  trail,  saw  the  blockage,  and  backed  up  three  times  until  the  run  was  halted 
for  max  backups. 


Figure  A-41.  Run  275-FCI-RIVET. 
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A.5  Scenario  5 


A.5.1  Autonomous  Mobility  Only 

All  three  RIVET  runs  ended  on  max  backups  immediately  upon  encountering  the  blockage  on 
the  initial  planned  path  (figure  A-42).  AM  only  would  see  the  blockage  of  the  original  path, 
back  up,  and  be  unable  to  find  a  plan  through  the  blockage. 


Figure  A-42.  Run  234-AM-RIVET. 


In  the  two  field  runs  (16.1  and  20)  with  the  AM-only  planner,  the  XUV  circled  repeatedly  in  the 
cul-de-sac,  trying  to  find  an  acceptable  way  along  the  planned  path  (figures  A-43  and  A-44). 
After  circling  three  or  more  times,  the  runs  were  e-stopped. 


Figure  A-43.  Run  16.1 -AM-Field. 
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Figure  A-44.  Run  20-AM-Field. 

A.5.2  Dynamic  Replanner 

In  the  three  RIVET  runs,  the  XUV  proeeeded  along  the  planned  route  and  saw  that  the  path  was 
bloeked.  It  attempted  to  go  through  the  alternate  route  nearby  to  the  north  but  saw  that  it,  too, 
was  blocked.  The  XUV  then  replanned  back  east  past  the  starting  point,  around  the  a  priori  trees 
and  south  toward  the  tank  trail  and  a  clear  path  to  the  goal  (figure  A-45).  These  runs  spent  49%- 
54%  of  the  time  in  HMP  at  the  cul-de-sac  while  generating  and  trying  alternate  plans. 


Figure  A-45.  Run  23-DR-RIVET. 
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The  five  field  runs  (figures  A-46-A-50)  resembled  very  closely  the  RIVET  runs.  The  field  runs 
tended  to  spend  additional  time  and  distance  exploring  alternate  routes  in  the  cul-de-sac, 
probably  because  of  the  highly  varied  vegetation  on/around  the  alternate  paths  out  of  the  cul-de- 
sac.  These  runs  spent  28%-53%  of  the  time  in  HMP  at  the  cul-de-sac  while  generating  and 
trying  alternate  plans. 
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Figure  A-46.  Run  30-DR-Field. 


Figure  A-47.  Run  27-DR-Field. 
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Figure  A-48.  Run  22-DR-Field. 


Figure  A-49.  Run  19-DR-Field. 
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Figure  A-50.  Run  18-DR-Field. 

A.5.3  Field  Cost  Interface 

In  the  three  RIVET  runs,  the  XUV  proceeded  along  the  planned  route,  saw  that  the  path  was 
blocked,  circled  several  times  while  attempting  to  find  a  way  through  the  original  and  alternate 
trails,  and  then  eventually  replanned  back  past  the  starting  point,  around  the  a  priori  trees  and 
south  toward  the  tank  trail  and  a  clear  path  to  the  goal  (figure  A-5 1). 


Figure  A-51.  Run  12-FCI-RIVET. 
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Two  of  the  five  field  nins  were  successful  in  finding  an  alternate  route.  Runs  23  and  28  (figures 
A-52  and  A-53)  were  successful  in  achieving  the  goal  but  were  100-200  m  longer  than  the 
corresponding  RIVET  runs  because  these  runs  spent  additional  time  and  distance  circling  in  the 
cul-de-sac  and  building  up  local  terrain  data  before  deciding  to  replan  back  past  the  starting 
point,  around  the  a  priori  trees  and  south  toward  the  tank  trail  and  a  clear  path  to  the  goal. 


Figure  A-52.  Run  23-FCI-Field. 


Figure  A-53.  Run  28-FCI-Field. 
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Three  of  the  runs  (17,  25.2,  and  21.1)  saw  the  blockage  and  spent  additional  time  and  distance 
circling  in  the  cul-de-sac,  but  FCI  never  generated  a  plan  back  toward  the  start  point  (figures 
A-54-A-56).  The  runs  were  ended  after  three  or  more  circles. 


Figure  A-54.  Run  17-FCTField. 


Figure  A-55.  Run  25.2-FCI-Field. 
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Figure  A-56.  Run  2 1 . 1  -FCI-Field. 


Two  of  the  five  field  runs  resembled  very  closely  the  RIVET  runs. 

A.6  Scenario  6 

A.6.1  Autonomous  Mobility  Only 

All  three  RIVET  runs  were  ended  on  max  backups  after  circling  and  repeatedly  trying  to  go 
through  the  blocked  area  (figure  A- 5  7). 


Figure  A-57.  Run  261-AM-RIVET. 
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In  the  one  field  run  with  AM  only,  the  XUV  circled  repeatedly  in  the  cul-de-sac,  trying  to  find  an 
acceptable  way  along  the  initial  planned  path  (figure  A-58).  After  circling  three  or  more  times, 
the  run  was  stopped. 


Figure  A-58.  Run  78-AM-Field. 

A.6.2  Dynamic  Replanner 

In  the  three  RIVET  runs,  the  XUV  proceeded  along  the  planned  route,  saw  that  the  path  was 
blocked,  attempted  to  go  through  the  alternate  route  nearby  to  the  north  but  saw  that  it  also  was 
blocked,  then  replanned  back  past  the  starting  point  and  discovered  a  path  through  the  trees  south 
toward  the  tank  trail  and  a  clear  path  to  the  goal  (figure  A-59).  These  runs  spent  40%^2%  of 
the  time  in  HMP  at  the  cul-de-sac  while  generating  and  trying  alternate  plans. 


Figure  A-59.  Run  258-DR-RIVET. 
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The  five  field  runs  (figures  A-60-A-64)  resembled  very  closely  the  RIVET  runs.  The  field  runs 
tended  to  spend  additional  time  and  distance  exploring  alternate  routes  in  the  cul-de-sac, 
probably  because  of  the  highly  varied  vegetation  on/around  the  alternate  paths  out  of  the  cul-de- 
sac.  These  runs  spent  3 1%-77%  of  the  time  in  HMP  at  the  cul-de-sac  while  generating  and 
trying  alternate  plans. 


Figure  A-60.  Run  77.1 -DR-Field. 


Figure  A-61.  Run  89-DR-Field. 
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Figure  A-62.  Run  86-DR-Field. 


Figure  A-63.  Run  84-DR-Field. 
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Figure  A-64.  Run  80-DR-Field. 

A.6.3  Field  Cost  Interface 

In  the  three  RIVET  runs,  the  XUV  proceeded  along  the  planned  route,  saw  that  the  path  was 
blocked,  circled  several  times  while  attempting  to  find  a  way  through  the  original  and  alternate 
trails,  and  then  eventually  replanned  back  past  the  starting  point  and  discovered  a  route  through 
the  trees  south  toward  the  tank  trail  and  a  clear  path  to  the  goal  (figure  A-65). 


Figure  A-65 .  Run  25 1  -FCI-RI VET. 
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All  five  field  runs  (figures  A-66-A-70)  were  suceessful  in  finding  an  alternate  route  similar  to 
the  runs  in  RIVET.  These  runs  spent  additional  time  and  distanee  eircling  in  the  cul-de-sac, 
building  up  local  terrain  data  before  replanning  back  past  the  starting  point  and  discovering  a 
path  through  the  trees  south  toward  the  tank  trail  and  a  clear  path  to  the  goal. 


Figure  A-66.  Run  88-FCI-Field. 


Figure  A-67.  Run  85-FCI-Field. 
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Figure  A-68.  Run  82-FCI-Field. 


Figure  A-69.  Run  79.1 -FCI-RIVET. 
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Figure  A-70.  Run  76-FCI-Field. 

All  five  field  runs  resembled  very  closely  the  RIVET  runs. 

A.7  Scenario  7 

A.7.1  Autonomous  Mobility  Only 

All  three  RIVET  runs  were  successfully  completed  as  the  XUV  sensed  the  blockage  and  turned 
around  180°  to  return  to  the  place  in  the  trail  where  an  alternative  trail  existed  to  the  goal  (figure 
A-71).  The  RIVET-modeled  foliage  around  the  blockage  allowed  this  maneuvering  (this  was  not 
expected). 


Figure  A-71.  Run  248-AM-RIVET. 
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The  field  runs  varied  eonsiderably.  Run  8  (figure  A-72)  completely  ignored  the  original  path  and 
skirted  to  the  south  of  the  cul-de-sac,  directly  to  the  goal.  Run  4  (figure  A-73)  found  the  cul-de- 
sac  but  backed  up  three  times  and  was  then  positioned  in  a  location  where  it  was  able  to  find  an 
open  path  along  the  trail  to  the  west  and  went  directly  to  the  goal.  Run  2  (figure  A-74)  was 
e-stopped  at  the  blockage  site,  run  10  (figure  A-75)  was  e-stopped  when  the  XUV  backed  into  a 
rock,  and  run  13  (figure  A-76)  was  ended  for  max  backups. 


Figure  A-72.  Run  8-AM-Field. 


Figure  A-73.  Run  4-AM-Field. 
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Figure  A-74.  Run  2-AM-Field. 


Figure  A-75.  Run  10-AM-Field. 
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Figure  A-76.  Run  13-AM-Field. 

A.7.2  Dynamic  Replanner 

In  the  three  RIVET  runs,  the  XUV  proeeeded  along  the  planned  route  and  saw  that  the  path  was 
bloeked.  HMP  directed  a  1 80°  turn,  and  the  XUV  followed  a  new  plan  back  the  trail  and  around 
the  southern  side  of  the  trees  to  the  goal  (figure  A-77). 


Figure  A-77.  Run  212-DR-RIVET. 
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In  the  field  runs,  after  the  blockage  was  encountered,  HMP  backed  the  XUV  the  entire  distance 
out  of  the  cul-de-sac  on  four  of  the  five  runs.  Two  of  those  runs  ended  in  completions  (figures 
A-78  and  A-79). 


Figure  A-78.  Run  1-DR-Field. 


Figure  A-79.  Run  6-DR-Field. 
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The  remaining  three  runs  were  able  to  eseape  the  eul-de-sae  but  were  unable  to  complete  the  run. 
A  suspected  navigation  problem  ended  run  7  (figure  A-80),  and  an  HMP  problem  ended  run  1 1 
(figure  A-81).  Run  15  ended  for  max  backups  (figure  A-82). 


Figure  A-80.  Run  7-DR-Field. 


Figure  A-81.  Run  1 1-DR-Field. 
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Figure  A-82.  Run  15-DR-Field. 


A.7.3  Field  Cost  Interface 

In  the  three  RIVET  runs,  the  XUV  proceeded  along  the  planned  route,  saw  that  the  path  was 
blocked,  circled  for  a  1 80°  turn  out  of  the  cul-de-sac,  and  followed  a  new  plan  back  the  trail  and 
around  the  southern  side  of  the  trees  to  the  goal  (figure  A-83). 


Figure  A-83.  Run  224-FCI-RIVET. 
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In  the  five  field  runs,  the  XUV  encountered  the  blockage  and  backed  up  three  times,  but  not  far 
enough  to  enable  the  XUV  to  see  the  unobstructed  path  to  the  goal  below  the  tree  line.  All  five 
runs  ended  with  max  backups  (figures  A-84-A-88). 


Figure  A- 84.  Run  14-FCI-Field. 


Figure  A-85.  Run  9-FCI-Field. 
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Figure  A-86.  Run  5-FCI-Field. 


Figure  A-87.  Run  3-FCI-Field. 
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Figure  A-88.  Run  12-FCI-Field. 


A.8  Excursion  Scenario  1.1 


A.8.1  Situation 


Scenario  1 . 1  was  the  same  as  scenario  1  except  that  the  NBC  area  was  removed  in  order  to  see  if 
the  DR  would  plan  to  use  the  low-cost  main  tank  trail  to  the  west  as  an  alternate  route  (figure 
A-89). 


Figure  A-89.  Scenario  1.1  description. 
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A.8.2  Dynamic  Replanner 

On  neither  run  did  the  DR  plan  a  path  that  utilized  the  main  tank  trail  to  the  west.  Run  116 
(figure  A-90  [right])  was  typical  of  the  DR  runs  31.3  and  42,  where  the  XUV  saw  the  blockage, 
returned  to  the  opening  on  the  right,  discovered  the  unmarked  trail  to  the  left,  and  proceeded 
slowly  down  the  unmarked  trail  under  HMP  control.  The  DR  generated  plans  that  directed  the 
XUV  back  to  the  original  trail,  beyond  the  blockage.  However,  the  XUV  was  unable  to  find  a 
clear  path  through  the  brush  and  trees,  and  eventually  the  DR  gave  a  plan  that  followed  the 
unmarked  trail  to  the  goal.  On  run  1 14  (figure  A-90  [left]),  the  XUV  was  e-stopped  when  the 
HMP  backed  it  into  an  unknown  area  of  high  brush  near  the  blockage. 


Figure  A-90.  Scenario  1.1  runs  with  no  NBC  blockage. 
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A.9  Excursion  Scenario  6.1 


A.9.1  Autonomous  Mobility  Only 

In  the  two  field  runs,  the  XUV  repeatedly  circled  in  the  cul-de-sac  and  never  came  close  enough 
to  the  unblocked  trail  in  the  north  to  see  the  opening  (figures  A-91  and  A-92).  After  circling 
three  or  more  times,  the  run  was  e-stopped. 


Figure  A-91.  Run  1 13.9- AM-Field. 


Figure  A-92.  Run  108. 9- AM-Field. 
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A.9.2  Dynamic  Replanner 

In  the  two  field  nins,  the  XUV  sensed  the  bloekage  in  the  original  path  and  quickly  generated 
and  executed  a  new  path  through  the  alternate  trail  to  the  north  (figures  A-93  and  A-94).  These 
runs  spent  37%  and  92%  of  the  time  in  HMP  at  the  cul-de-sac  while  generating  and  trying 
alternate  plans. 


Figure  A-93.  Run  106.9-DR-Field. 
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Figure  A-94.  Run  111.9-DR-Field. 
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A.9.3  Field  Cost  Interface 


In  the  two  field  nins,  the  XUV  sensed  the  bloekage  in  the  original  path  and  quickly  generated 
and  executed  a  new  path  through  the  alternate  trail  to  the  north  (figures  A-95  and  A-96).  This 
occurred  quickly  and  seamlessly  on  both  runs. 


Figure  A-95.  Run  1 12.9-FCI-Field. 


Figure  A-96.  Run  107.9-FCI-Field. 
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A.10  Excursion  Scenario  6.2 


A.10.1  Field  Cost  Interface 

As  expected,  on  both  runs,  the  XUV  exited  the  cul-de-sac  area  after  circling  twice  near  the 
blockages  (figures  A-97  and  A-98).  One  run  was  completed  and  the  second  run  was  e-stopped 
when  the  XUV  ran  into  undetected  and  unmarked  water  after  exiting  the  cul-de-sac. 


Figure  A-97.  Run  110.9-FCI-Field. 


Figure  A-98.  Run  109.9-FCI-Field. 
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Appendix  B.  Experimenter’s  Observation  of  All  Runs 


This  appendix  appears  in  its  original  form,  without  editorial  ehange. 
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Source 

Run 

Run 

Scenario  # 

Planner 

Outcome 

Reason 

RC 

Comments 

JP  Comments 

MC  Comments 

SH  Comments 

BB  Comments 

Field 

1 

1 

7 

DR 

Complete 

Great  run.  HMP  plan  on  red  in 
the  beginning 

Start  backup  out  of  road  -  HMP 

Backed  out  from  barrels.  Turned 
toward  goal.  HMP  backed  it  out  of 
cul-de  sac.  Complete 

Field 

2 

2 

7 

AM  only 

DNF 

Estop 

e-stopped  before  detecting 
barrels,  went  around  and  into 
bushes.  E-stopped. 

Changed  backup  to  10,20,30m. 

Rerun.  Backup  10,  20  (almost  to 
road).  Backup  -  starts  to  left.  Estop  - 
going  where  it  can’t  go.  End  of 
mission  called  by  Test  Director  (TD) 

Turn  up  the  hill.  E-stop  at  barrels. 
Re-ared.  Two  backups,  went 
forward  and  stopped.  Backed  all  the 
way  out  to  road.  Turned  toward 
goal.  Eooped  left.  Headed  back  to 
start  point.  Killed  it. 

Field 

3 

3 

7 

FCI 

DNF 

Stuck-max 

backups 

max  backups 

After  turn,  stop  to  look  around  - 
continued  forward.  Backkup,  then 
goes  forward.  Backup  into 
intersection,  continues  forward.  Stuck 
-  end  of  mission  by  TD 

BU  from  barrels  almost  to  road. 
Again  should  have  seen  alternative 
path.  Stuck.  End  of  mission. 

Field 

4 

4 

7 

AM  only 

Complete 

admin  e-stop  twice  for  safety. 
Plan  complete 

Backup,  backup,  take  same  road  to 
right.  Estop  (admin)  to  make  sure  its 
clear,  rearm.  Backup  into  interaction. 
Turn  left  onto  left  path.  Turn  to  right 
-  Estop  (admin).  Rearm,  resume. 
Complete 

Estop.  Rearm.  30  m  BU  to  road. 
Positioned  correctly.  Cut  across  up 
to  road.  Complete. 

Field 

■ 

5 

7 

FCI 

DNF 

Stuck-max 

backups 

max  backups 

Backup,  backup  into  intersection. 

Goes  back  up  to  the  right.  Stuck  - 
Max  Backups 

Up  to  barrels.  BU  to  road.  Par 
enough,  but  went  back  up  to 
barrels.  Stuck  max  Backups. 

Field 

6 

6 

7 

DR 

Complete 

admin  e-stop.  Plan  complete 

“R:26”  on  status  bar  on  displays 
showing  replanning  using  DR.  “U: 
Need  plan”  on  status  bar  shows 
requesting  plan  from  HMP.  Estop 
(Admin)  to  get  “eyes  on”  XUV. 
Complete  [Note:  after  run,  Mike 
removes  rock  from  middle  of  road] 

5  m  from  barrels  stop.  BU  3/4  way 
down  and  move  forward.  BU  res  of 
way  down.  Off  to  goal.  Admin  e- 
stop  clear.  Crossed  up  from  road  to 
path.  Complete. 

Field 

7 

7 

7 

DR 

DNF 

Nav  problem 

E-stopped.  Saw  path  blocked 
ahead  (wrong)  and  turned  left 
(bushes)  for  alternative. 

Possibly  issues  with  IRU 
calibration 

HMP  -  making  K  turn.  Estop  - 
because  of  terrain.  Backup.  End  of 
mission  -  called  because  could  not 
assess  terrain  in  front  of  bot  (called 
by  TD).  Turned  to  left  off  road 
because  road  looked  completely 
blocked  due  to  “position  error” 

To  barrels.  Then  partway  back 
down.  Charge  woods  to  left.  Turned 
around.  Out  to  road  forward. 

Turned  to  goal.  Overshot  crossover. 
Turned  left  into  trouble  in  high 
brush.  (Suspect  run  after  lunch, 
NAV) 

In.  IRU  was  a  problem  shortly 
afterward.  Inconclusive  here. 

Field 

8 

8 

7 

AM  only 

Complete 

Didn't  turn  right! ! !  no  idea  why. 
Possibly  issues  with  IRU 
calibration 

Went  straight  ahead  then  stopped 
when  within  20m  of  endpoint  (did  not 
make  right  tum)Complete  [showed 
that  the  bot  was  within  path  tolerance 
of  30m  so  did  not  show  “out  of  path” 
error]  [could  be  that  going  fast  and 
debating  whether  to  go  left  or  right; 
maybe  going  too  fast  to  make  right 
turn] 

Overshot  the  road  to  cul-de  sac 
completely.  Straight  to  goal. 

In.  IRU  was  a  problem  shortly 
afterward.  Inconclusive  here. 

Field 

9 

9 

7 

FCI 

DNF 

Stuck-max 

backups 

Engine  stalled.  Max  backups 

Stalled  coming  up  hill.  Backup  10, 

20,  30.  Operator  Max  Backups  - 
Stuck  -  End  of  mission  [at 
intersection,  RCI  couldn’t  find  a  path 
to  the  left] 

Max  backups. 

In.  IRU  was  a  problem  shortly 
afterward.  Inconclusive  here. 

Field 

10 

10 

7 

AM  only 

DNF 

Estop 

Estop-backed 
into  big  rock 

backup  3  times  and  was  stuck 

Saw  barrels,  backed  up,  went  in 
toward  barrels  again,  backed  up,  E- 
Stop  (backing  into  rock).  Stuck, 

E.O.M. 

Field 

11 

11 

7 

DR 

DNF 

Estop 

Estop-HMP 
stuck  on 
obstacles 

knocked  over  3  barrels  on 
hmp.  11:38  End  of  mission  hmp 
could  not  plan  off  of  obstacle 

Saw  barrels,  backup,  backup,  HMP 
dance,  k-turn,  accidental  e-stop, 

XUV  is  clear  of  brush  but  hesitating, 
backup,  came  out  of  trail,  turned  right 
towards  alternative  path  but  overshot 
the  turn,  accidental  e-stop,  HMP 
planning  for  long  time,  HMP  cannot 
find  path  (XUV  is  "sitting"  on 
obstacles),  E.O.M. 

HMP  Pailure  to  get  off  obstacle. 

Field 

12 

12 

7 

FCI 

DNF 

Stuck-max 

backups 

Stuck-max 

backups 

backed  up  3  times  and  was  stuck 

Stalled  going  up  hill,  backup,  backup, 
backup.  Stuck  (Max  backups), 

E.O.M. 

Field 

13 

13 

7 

AM  only 

DNF 

Stuck-max 

backups 

Stuck-max 

backups 

stuck  max  backup 

Backup,  Backup,  Backup,  accidental 
e-stop,  max  backups,  E.O.M. 

Field 

14 

14 

7 

FCI 

DNF 

Stuck-max 

backups 

Stuck-max 

backups 

3  backups  stuck 

Accidental  e-stop,  backup,  backup, 
backup,  backup,  backup.  Stuck, 

E.O.M. 

Field 

15 

15 

7 

DR 

DNF 

Stuck-max 

backups 

Stuck-max 
backups,  filled 
in  road  path 

Eor  some  reason  the  map  on 
idisp  got  blocked  so  the  bot 
could  not  plan  localy  to  the 
goal,  stuck  maxbackups 

Saw  barrels,  HMP  turned  XUV 
around,  backup,  admin  stop,  came 
out  of  trail,  overshot  slightly  on  turn, 
HMP  maneuvering  to  take 
alternative,  accidental  e-stop,  backup, 
backup,  backup,  HMP  maneuvered 
away  from  path  towards  our  location, 
HMP  backed  up  and  oriented  XUV  to 
face  alternative  path,  started  to  go. 
Stuck,  E.O.M. 

Field 

16 

16 

5 

AM  only 

DNF 

Estop 

nicely  drove  into  bushes.  E- 
stopped 

Estop  when  went  through  middle 
pathway.  Added  more  blockage  and 
rerun 

Closed  on  barrels.  BU  10  m. 

Turned  north.  Charged  brush. 

Estop. 

Field 

16.1 

216 

5 

AM  only 

DNF 

Estop 

placed  HMWVV  into  bushes, 
run  into  bushes.  E-stoped.  killed 
log.  Restarted  log.  Continuation 
of  Run  16a.  11:11  E-stopped. 
Unable  to  find  way  out. 

Estop  (admin)  to  get  truck  out  of  the 
way.  Estop  -  heading  into  bushes  on 
left.  Stop  after  3 -circle  rule 

Stopped  at  barrels.  Move  north. 
Circe.  Stop.  Three  to  four  circles 
and  we  stopped  it. 

same  as  16.2 

Field 

17 

17 

■ 

FCI 

DNF 

11:17  bot  stuck  after  one 
backup,  admin  e-stop,  seing 
things  on  the  road,  went 
exploring...  e-stopped.  Possibly 
issues  with  IRU  calibration, 
engine  not  sounding  right.  E- 
stop.  continue  from  here,  didn't 
find  alternative  goal.  Stopped 

Approached  middle  path,  stalled, 

“Bot  Stuck”,  resumed.  Estop  for 
safety  (too  near  HMMWV).  Estop  for 
safety  (too  near  Robert).  Resume. 
Estop-  stalled  -  stuck  on  hill  and 
ditch.  Resume  -  circling  around 
many  times.  Suspend  -  End  of 
mission  by  TD 

Multiple  loops.  Ultimately  we 
stopped  it. 

Field 

18 

18 

5 

DR 

Complete 

64  plans,  looks 
like  lostNAV, 
got  past  the 
short  end  point 

1 1 :54  e-stopped  before  puddle. 
Teleop  around.  Continue.  11:59 
AM  sign  on  the  road.  E-stop  to 
pull  it  out 

Poked  around  and  then  headed  out. 
Teleop  around  puddle,  [dark  blue 
dotted  line  appears  every  time  there  is 
a  new  C2  plan] 

Successful  recognition  of  blockage. 
Routed  around  apriori  data.  Went  to 
goal. 

Field 

19 

19 

5 

DR 

Complete 

goto  haven't  moved.  1:38  admin 
e-stop.  1:47  redraw  puddle.  1:53 
mission  complete 

Estop  (admin)  for  safety  near 
HMMWV.  Resume.  Poked  around 
and  then  went  back  out  of  finger. 

Estop  for  water  hole 

Charged  into  brush,  got  its  bearing 
and  looped  around  th  apriori. 

Field 

20 

20 

5 

AM  only 

DNF 

2:10  pm  accidental  e-stop. 
2:13PM  unable  to  get  out 

Accidental  Estop.  Poking  around. 

End  of  mission  -  3  circle  rule  (AM 
stuck  in  cul-de-sac) 

Many  circles  and  stopped  it. 

Field 

21 

21 

■ 

FCI 

DNF 

Not  recovering  well  from  Estop. 
Thinking  the  NBC  location  is 
expanding  and  screwing  up  planning 

Circled  many  times  but  turned  out 
of  the  football  field.  Then  got  to 
puddle  (barrels  added)  and  stopped. 
Brought  it  back  an  AM  cut  across. 

NBC  took  a  good  run  and  made 
it  bad.  We  need  Marshal  to 
weigh  in  on  this. 

Field 

21.1 

221 

5 

FCI 

DNF 

Estop 

e-stopped  for  puddle,  unable  to 
resume 

Circled  around.  Estop  on  puddle  at 
the  point.  Einish  run. 

Field 

22 

22 

5 

DR 

Complete 

removed  puddle  from  SMI. 
Placed  barrels  in  front  of 
puddle,  accidental  e-stop. 
Complete 

Accidental  Estop.  Stalled  going  up 
hill.  Complete  [only  way  operator 
knows  what  mode  is  by  the  replan] 

Quickly  sized  up  barrels  and 
blockage  and  headed  out  of  finger. 

Field 

23 

23 

5 

FCI 

Complete 

stopped  for  puddle,  going 
slowly  (2.5m/s).  Plan  complete 

JP  walked  in  front  of  XUV  to  create 
obstacle  so  it  won’t  go  thru  bigger 
puddle 

Circled  once  or  twice.  JP  walked  in 
front  to  create  obstacle  at  puddle  to 
south  and  west  of  barrels.  PCI 
found  its  way  out  to  goal. 

Field 

25 

25 

5 

FCI 

DNF 

Estop 

estop-water 

had  to  estop  the  bot  because  it 
was  heading  to  a  huge  puddle 

Drove  over  cones  in  center  area, 
backup,  drove  into  mud  puddle, 
backup,  circled  round,  drove  into 
cones  in  center  area,  backup,  drove 
out  of  cul-de-sac,  turned  around  and 
came  back  (reportedly  Prim  data  is 
scrolling  off  of  map  and  causing  this 
bungie  cord  effect),  E.O.M. 

Field 

25.1 

225 

5 

FCI 

DNF 

Circled 

circled 

had  to  estop  but  because  of 
puddle  again,  idisp  capture 
failed-had  to  restart  idisp 

Backup,  headed  into  mud  over  cone. 
E-stop,  E.O.M.  (moved  barrels  (3)  to 
block  puddle) 

Field 

25.2 

226 

■ 

FCI 

DNF 

Circled 

circled,  exited 
3x,  came  back 

fci  just  oscillated  back  and  for 
and  would  not  leave  culdesac 

Drove  over  cones  in  center  area, 
backup,  drove  into  mud  puddle, 
backup,  circled  round,  drove  into 
cones  in  center  area,  backup,  drove 
out  of  cul-de-sac,  turned  around  and 
came  back  (reportedly  Prim  data  is 
scrolling  off  of  map  and  causing  this 
bungie  cord  effect),  E.O.M. 

Field 

27 

27 

5 

DR 

Complete 

good  at 
culdesac, 
exitted 

succussfully  reached  road  ended 
mission  due  to  lack  of  range 
time 

Eooked  right  and  backed  up,  looked 
left,  backup,  looked  in  the  center 
area,  backup 

Field 

28 

28 

5 

FCI 

Complete 

got  to  road, 
circled,  EOM 

obstHeight=0.5.  successfully 
turned  around.  Once  it  got  to 
road,  headed  in  the  wrong 
direction,  turned  around  and 
saw  chase  vehicle,  e-stopped 

0.5m  obstacle  height,  XUV  inspected 
blocked  path,  circled  round  a  couple 
of  times,  chose  alternative  path  out  of 
sul-de-sac 

Three  circles,  one  back  up.  Went 
out ,  turned  to  road.  One  more 
circle  at  the  road. 

Field 

30 

30 

5 

DR 

Complete 

hmp  on  obststacle.  paused  as  it 
entered  the  road  back,  mission 
complete 

XUV  detected  obstacles,  assumed 
full  blockage,  replanned,  maneuvered 
on  alternative  path  out  of  cul-de-sac 

Eorward  and  back  several  times 
probing  varrels.  Plan  to  left.  Einally 
HMP  and  DR  showed  a  way  out. 

Field 

31 

234 

1 

DR 

DNF 

could  not  find  its  way. 

Took  right  turn  immediately,  admin 
stop,  backup  5m,  10m,  15m,  E.O.M. 
declared  -  recommended  to  reduce 
barrel  height  (obstacle  height  =  1.5m) 

Field 

31.1 

233 

1 

DR 

DNF 

HMP  backed 
into  brush 

reduced  barrel  height  and  started 
closer  to  the  start  point,  bot 
backing  up  into  unknown  in 
hmp.  10:35  stopped 

XUV  went  further  down  path,  saw 
barrels,  then  invoked  HMP,  drove 
backwrds  through  brush  along  new 
path.  E-stop,  E.O.M. 

Went  into  woods  to  right.  HMP 
moving  it  in  that  direction. 

Field 

31.2 

231 

1 

DR 

DNF 

new  barrel 
locations+lOm: 
saw  alt  rt  as 
blocked 

starting  farther  back  and  barrles 
moved,  failed  to  find  way  out 

Saw  obstacles,  stopped  adjacent  to 
opening,  hesitated  because  HMP 
could  not  find  valid  path,  backup  5m, 
turned  right  at  opening,  took  left  at 
next  opening,  backed  up,  turned  right 
and  drove  into  brush  (searching  for 
path),  E.O.M. 

comments 

Field 

31.3 

232 

1 

DR 

Complete 

new  barrel 
locations+lOm 

obst  height  1.5.  complete:  but 
goal  moved  before  ending  and 
had  to  pause 

E-stop,  backup,  HMP  backed  XUV 
down  alternative,  backed  over  tall 
brush  and  small  trees,  turned  around, 
went  forward  over  small  tree,  round 
road,  maneuvered  to  goal 

Backed  into  second  path. 

Eventually  turned  down  altematie 
path  and  completed. 

look  at  JP 

Field 

32 

32 

1 

FCI 

Complete 

3  Bus  for  60m, 
found  alt  rt, 
direct  to  goal 

obst  height=1.5, 
coverheight=3.0.  plan  complete 

Obstacle  height  =  0.5m,  saw  barrels, 
veered  right,  backup  10m,  backup 
20m,  backup  30m,  cut  corner  to 
alternative  path.  E-stop,  maneuvered 
to  goal 

10  m  BU,  20  m  BU,  30  m  BU. 
Admin  e-stop.  Eound  path  to  right. 
(Effectively  a  40-50  m  backup.) 

Did  nt  do  NBC.  60  m  BU  total. 

Field 

33 

33 

1 

AM  only 

DNF 

Stuck-max 

backups 

ran  thru  barrels 
3x,  3BUs, 
stuck  on  log 

changing  obst  height  to  1.4. 
stuck  backing  up 

Obstacle  height  =  1 .4m,  saw 
blockage,  backup,  dove  left,  backup, 
drove  forward,  backup  into  trees. 
Stuck  (physically),  E.O.M. 

BU  got  physically  stuck.  Height  1.4 

m. 

Field 

34 

34 

1 

FCI 

DNF 

Stuck-max 

backups 

backed  up 
~40m,  max 
backups 

10:34  stuck  max  backups 

Saw  barrels,  backup,  dove  left, 
backup  20m,  backup  30m,  backed 
into  trees.  Stuck  (Max  backups), 

E.O.M. 

Stuck  max  BU.  Only  came  back  40 
m  total. 

Field 

35 

35 

1 

DR 

Complete 

seoCoverHeight=3.0.  1:35 
complete 

Saw  barrels,  backed  up,  HMP 
maneuvered  XUV  to  path  on  right, 
XUV  followed  path  and  took  left 
down  alternative,  XUV  is  hunting 
and  pecking  to  back  to  original  path 
(reportedly  the  global  planner  did  not 
consider  the  barrels  to  be  a  full 
blockage),  XUV  went  through  brush 
and  went  behind  barrels  to  join  the 
road,  admin  stop,  maneuvered  to  goal 
[Planning  to  get  to  road  beyond  the 
barrels.] 

Complete. 

o 


Field 

38 

38 

1 

DR 

DNF 

Stuck-max 

backups 

found  alt  rt, 
HMP  backed 
into  brush 

1:50  bot  stuck.  AM  didn's  see  a 
path  forward 

Saw  barrels,  stopped,  backed  up, 
turned  right  on  alternative,  turned  left 
to  go  down  path  (looking  to  get  back 
on  original  path),  XUV  backed  up 
(but  was  to  right  off  path)backed  into 
small  trees.  E-stop,  Resume,  Stuck, 
E.O.M. 

Backed  into  small  trees.  HMP 
failure.  Incomplete. 

HMP  Eailure 

Field 

39 

39 

1 

FCI 

DNF 

Stuck-max 

backups 

stuck  max  backups 

Saw  barrels,  stopped,  trying  to  go 
around  to  left,  E-Stop,  Resume, 
backup,  backup  20m,  backup  30m, 
Stuck  (Max  backups),  E.O.M. 

Incomplete 

Field 

42 

42 

1 

DR 

Complete 

tried  to  return 
to  left  rd, 
continued  to 
right  to  goal 

11:01  plan  complete 

Saw  barrels,  stopped,  HMP  backup 
and  k-turn,  came  back  along  original 
path,  turned  left  on  alternative  path, 
left  again,  cut  left  into  brush,  backup, 
backup.  Admin  stop  (to  move  truck). 
Resume,  backup,  backup,  took  left 
shorcut  again,  backup  (near  trail), 
backup,k  took  path  o  right  to  road. 

Complete 

Ilustration  Run 

Field 

43 

43 

1 

DR 

DNF 

Estop 

HMP  backed 
into  brush  on 
k-turn  at 
barrels 

hmp  vehicle  on  red.  ACC2 
planning  through  obstacles...?? 
backup  path  not  safe 

Saw  barrels,  backup,  HMP  backed 
into  unknown  terrain.  E-stop, 

Resume,  backing  into  unknown 
terrain  again,  E.O.M. 

Incomplete 

HMP  Problem  but  DR  did  not 
make  a  strong  case. 

Field 

46 

46 

2 

FCI 

DNF 

broke  steering 

lost  steering 

hit  rock  and  broke  steering 

XUV  bounced  down  path,  saw 
blockage,  backed  up  in  a  zig  zag 
fashion,  backed  into  brush,  backed  in 
a  circle,  E-Stop,  E.O.M. 

Scenario  2  not  what  was 
intended. 

Field 

46.1 

146 

2 

FCI 

DNF 

obstheight=1.4.  coverheight=3. 
speed  3.5  m/s.  bot  backed  up 
from  the  begginning 

XUV  immediately  backed  up,  XUV 
hit  barrels 

Field 

46.2 

147 

2 

FCI 

DNF 

Stuck-max 

backups 

AM  stuck  becau  se  of  e-stop. 
Stuck 

XUV  saw  the  blockage  (but  bumped 
barrels),  backup  10m,  backup  20m, 
XUV  found  alternative  path,  E-Stop 
(went  right  into  brush).  Resume, 
backup  10m,  Admin  stop  (to  allow 
truck  to  clear).  Resume,  Stuck 
(Operator  Stuck),  E.O.M. 

Field 

47 

47 

2 

AM  only 

Complete 

knocked  down  two  barrels,  plan 
complete 

XUV  saw  blockage  but  bumped 
barrels  as  it  stopped,  backup,  took 
right  path,  backup.  E-stop  (rock), 
achieved  goal 

Field 

48 

48 

2 

DR 

Complete 

not  running  hmp.  starting  it 
mid-run 

XUV  saw  barrels  and  bumped  the 
first  as  it  stopped,  backup  10m, 
backup  20m,  took  right  alternative 
path,  found  goal 

Field 

49 

49 

2 

DR 

Complete 

1 1 :24  plan  complete 

XUV  ran  into  2  barrels.  E-stop, 
Resume,  Backup,  HMP  looking  for 
path,  trying  to  go  left,  nosing  into 
trees,  backup,  pushing  trees,  backup, 
took  right  path,  got  near  goal  and 
tried  to  cut  corner,  HMP  backup  and 
dancing,  found  goal,  E.O.M. 

Field 

50 

50 

2 

FCI 

Complete 

hit  barrels  again.  fwOOO.sav. 
plan  complete! 

XUV  charged  barrels.  E-stop, 

Resume,  backup  10m,  backup  20m, 
turn  right  on  alternative,  turned  left 
into  brush,  backup,  backup  20m, 
continues  along  alternative  path, 
went  across  main  raod  toward 
helipad,  backup  10m,  backup  20m, 
bacWp  30m,  maneuvered  to  goal 

Field 

51 

51 

2 

AM  only 

Complete 

hit  barrels  again,  e-stop,  plan 
complete  (too  early?) 

XUV  bumped  barrels,  backup  10m, 
backup  20m,  took  alternative  path  to 
right,  backup  30m,  admin  stop. 
Resume,  admin  stop.  Resume,  cutting 
left,  backup  10m,  cut  right  to  road, 
then  to  goal 

Field 

52 

52 

2 

AM  only 

Complete 

plan  complete  (after  adjusting 
steeting  ...  seerun53e) 

AM  saw  barrels  and  turned  to  right 
w/out  entering  cul-de-sac,  backup 

10m,  maneuvered  to  goal 

Field 

53 

53 

2 

DR 

Complete 

plan  complete 

Immediately  took  right  turn  backup. 
Admin  stop,  proceeded  to  goal 

Field 

53.9 

153 

2 

DR 

Complete 

after  adjusting  steering,  laser 
pan.  obst  height  1.3. 
seoAngleOffsetPan=-0.0667. 
plan  complete.  HMP  not 
working  well 

XUV  saw  barrels  and  stopped  short, 
backup  5m,  backup  2m,  backup  2m, 
backup  3m,  backup  5m,  took 
alternative  path  to  the  right.  E-stop 
(Rock),  Resume,  reached  goal 

Field 

54 

54 

2 

FCI 

Complete 

plan  complete 

Saw  barrels  and  immediately  headed 
right,  stopped,  resumed,  found  goal 

Field 

55 

155 

2 

FCI 

DNF 

placing  HMWWV  in  path  to 
right,  found  gap. 

Saw  barrels  and  stopped  in  cul-de- 
sac,  backup  10m,  backup  20m,  E- 
stop  (backed  too  close  to 
HMMMWV),  backup  30m,  drove 
into  cul-de-sac  and  stopped  short, 
backup  10m,  E-stop  (tried  to  cut  in 
front  of  HMMWV,  E.O.M.  Place 
HMMWV  to  block  right  alterntive 
path.  Stopped  in  cul-de-sac,  backup, 
backup,  headed  toward  left  path, 
backed  up  into  trees.  E-stop,  Resume, 
backup  into  trees.  E-stop,  Max 
backups.  Resume,  forward  toward 
cul-de-sac,  backup  into  trees.  E-stop, 
E.O.M. 

Field 

56 

56 

2 

DR 

DNF 

backed  up  into  unknown 

E-stop  (headed  for  HMMWV), 
Resume,  entered  cul-de-sac,  backup, 
HMP  dancing,  backing  up  into 
unknown.  Admin  stop.  Resume, 
backing  into  trees  again,  E.O.M. 

Field 

57 

57 

2 

AM  only 

DNF 

too  close  to  HMMWV 

E-stop  (headed  for  HMMWV), 
Resume,  entered  cul-de-sac.  E-stop 
(charging  barrels),  backup  towards 
HMMWV  (E-stop),  Resume,  backing 
close  to  HMMWV,  E-stop,  E.O.M. 

Field 

61 

61 

3 

FCI 

Complete 

3  BUs  and 
around  to  right 

obst  height  1.5.  cover  height 

3.0.  original  prior  data, 
complete!!! 

XUV  went  up  hill,  saw  barrels, 
backup,  backup  20m,  E-stop  (headed 
for  truck),  found  alternative  path, 
drove  to  goal 

ECI  on  3rd  Back  up  30  m.  Eound 
path  to  right. 

Field 

62 

62 

3 

DR 

DNF 

did  not  see  1 
barrel,  went 
into  weeds, 
HMP  problem 

hmp  backed  into  bush,  bot  not 
moving,  canceled  mission 

XUV  drove  up  hill  and  saw  an 
opening  in  left  most  barrel,  backup, 
HMP  tried  to  execute  k-tum,  XUV 
trying  to  go  around  barrels,  HMP 
backed  XUV  into  brush,  E.O.M 

Right  up  to  barrel.  Move  to  left. 
Stopped.  Repeat.  HMP  backed  way 
up  into  woods  to  north.  HMP  in 
weird  mode. 

HMP  Eailure  but  DR  not 
demonstrated  yet. 

Field 

63 

63 

3 

AM  only 

DNF 

Stuck-max 

backups 

3  BUs,  stuck 

2:55pm  stuck 

Stopped  at  barrels,  backup  10m 
backup  20m,  backup  30m,  looked 
right  and  went  up  hill  again.  Max 
backups,  E.O.M. 

AM  to  barrels.  Backups  10,  20,  30. 
Could  have  seen  alternative  route 
but  did  not. 

Field 

64 

64 

3 

AM  only 

DNF 

Stuck-max 

backups 

3  BUs,  stuck 

stuck  b/c  engine  stalled,  re- 
executing.  3:02  pm  stuck 

Backup  10m,  stuck,  backup  10m, 
backup  20m,  backup  30m,  Stuck 
(Max  backups),  E.O.M. 

All  the  way  to  barrels.  10  m  BU 
(actually  8),  another  10  m,  then  20, 
then  30.  Stuck  max  BU. 

Field 

65 

65 

3 

DR 

DNF 

Estop 

tried  apriori  rd 
to  left,  crossed 
trail,  couldn't 
find  alt  rt 

3:20pm  hmp  not  moving,  restart 
hmp.  3:24  vehicle  started 
exploring  too  far  from  path,  e- 
stopped. 

Engine  stalled,  detected  partial 
blockage,  backup,  HMP  backed 
down  hill,  turned  left  and  took  false  a 
priori  road,  admin  E-stop,  XUV  sees 
partial  blockage  in  road,  turned 
round,  drive  over  to  alternative  path 
(on  right),  backup.  Stuck  in  HMP 
mode,  killed  HMP,  Resume 

Stopped  right  at  barrels.  Backing 
straight  slowly  down  the  hill. 
Oscillate  back  and  forth  for  a  while. 
Eook  for  false  apriori  road.  Ealse 
apriori  blocked.  Overshot  trail  to 
north.  Backed  up  toward  trail.  Up 
path.  Explore  to  right.  Brush  too 
thick. 

Eilm  worthy. 

Field 

66 

66 

3 

FCI 

DNF 

Stuck-max 

backups 

3  BUs,  stuck 

3:35pm  stuck  (max  backups) 

Saw  barrels,  stopped  just  shy, 
backup,  accidental  e-stop,  backup 
20m,  backup  10m,  backup  20m, 
backup  30m,  Stuck  (Max  backups), 

E.O.M. 

All  the  way  to  barrels.  Backups  of 
10,  20  m.  New  set  of  Bus  to  10,  20, 
30  m.  Max  backups. 

Field 

68 

68 

3 

DR 

DNF 

Estop 

backed  into 
weeds  on  left, 
HMP  problem, 
backed  into 
tree 

stuck  into  tree 

Saw  barrels,  backup,  HMP  k- 
turn/backup  (filling  in  blockage), 
backup  (filling  in  more  of  blockage), 
HMP  is  stuck  (won't  move),  backing 
into  trees.  E-stop,  E.O.M. 

All  the  way  up  to  barrels.  Slow 
back  up  progression  down  hill. 
Eorward  and  back,  filling  map. 

HMP  Back  up  in  funny  mode.  End 
mission. 

HMP  Eailure 

o 


Field 

69 

69 

3 

FCI 

Complete 

Stuck-max 

backups 

3  BUs  and 
around  to  right 

stuck  max  backups,  almost 
cleared,  resumed,  plan 
complete!!! 

Saw  barrels,  backup  10m,  backup 
20m,  backup  30m,  turned  right  on 
alternative  path.  Stuck  (Max 
backups).  Resume,  found  alternative 
and  drove  to  goal 

Back  up  few  times.  Turn  to  right . 
Stuck  max  BU.  Resume  and  found 
hill.  Overruled  max  back  up  in  this 
case. 

Field 

72 

72 

3 

DR 

DNF 

Estop 

ran  thru  barrel 

hit  barrels  and  kept  moving 

XUV  climbed  hill,  hesitated  (but 
didn't  stall),  continued  through 
barrels.  E-stop,  E.O.M. 

Right  up  to  barrels.  Went  through 
them.  End  of  mission.  Thought 
EADAR  was  looking  over  top  of 
barrels. 

Field 

76 

76 

6 

FCI 

Complete 

Estop  -  cause?  Complete  mission 
[***Note  that  run  number  was 
incorrect  wrt  the  original  run  sheet, 
but  went  with  the  run  #s  listed  here] 

Eound  its  way  out. 

Field 

77 

77 

6 

DR 

DNF 

Estop 

Plgr  fill  changed,  found  opening 
through  the  bushes 

Estop  -  into  path  -  rearm  to  see  if  it 
would  back  up,  it  didn’t.  End  of  run 
by  TD. 

Bot  found  a  hole  to  right  of  upper 
trail.  Redo. 

Field 

77.1 

277 

6 

DR 

Complete 

safety  e-stop.  Not  enough 
blockage,  hmp  didn't  move  for  a 
while  .  e-stopped  -  going 
through  ditch 

Complete.  Estop  -  tried  to  go  thru 
path  at  right.  Estop  -  stuck  Several 
other  Estops.  End  of  mission  by  TD  - 
seen  the  most  interesting  part  of  the 
run 

Einally  got  out  of  cul-de  sac.  Got 
stuck  in  woods.  Got  out.  Then  out 
of  cul-de-sac.  Ran  into  water 
almost.  Called  end  of  mission. 

Field 

78 

78 

6 

AM  only 

DNF 

stopped,  going  in  circles 

3-circle  rule.  End  of  mission  called 
by  TD 

Many  loops.  Einally  killed  it. 

Field 

79 

79 

6 

FCI 

DNF 

Estop 

e-stopped  because  of  large 
puddle 

Ran  circles.  Headed  out  of  finger,  but 
then  turned  around  again  for  a  big 
circle.  End  of  mission  called  by  TD 

Run  to  water. 

Field 

79.1 

279 

6 

FCI 

Complete 

seoMinObstHeightE  ast/  S  lo  w=0 . 
5.  got  out  of  cul-de-sac...  maybe 
faster??.  1 1 :02  stopped  because 
of  puddle,  drove  nicely  through 
narrow  path 

Reduce  obstacle  height  to  0.5m.  (on 
previous  runs,  height  has  been  0.75) 

Field 

80 

80 

6 

DR 

Complete 

obstHeight  back  to  0.75.  1 1 :33 
back-forth  w/o  moving.  1 1 :42 
plan  complete 

Obstacle  height  at  0.75m.  Complete 

Complete 

Field 

82 

82 

6 

FCI 

Complete 

obstHeight  0.5.  12:02  plan 
complete 

Inconclusive.  Obstacle  height  0.5m.  3 
times  around 

Complete 

Field 

84 

84 

6 

DR 

Complete 

2:01  global  path  not  blocked. 

2:19  goto  no  plan  message  but 
everything  fine.  2:20  plan 
complete 

E-stop  to  prevent  from  heading  over 
bank.  E-stop  (went  over  barrels)  - 
walked  in  front  of  XEIV  to  create 
obstacle,  hit  a  branch-barrel  combo, 
stopped,  replanned  out  of  culd-de-sac 

More  lost  than  usual.  Had  to  walk 
in  front  at  end  barrels  to  fill  map. 

Got  out  of  field  to  the  side.  Cutting 
through  to  the  road. 

Field 

85 

85 

6 

FCI 

Complete 

reduced  prior  data,  obst  height 
0.5.  10:12  AM  e-stopped  before 
puddle.  Otherwise  good  run 

0.5m  obstacle  height,  XEIV  made 
several  turns,  went  away  from 
blockage,  charged  safety  truck,  made 
some  more  passes,  assumed  full 
blockage,  took  alternative  path  out  of 
cul-de-sac 

four  circles,  one  "S",  two  ellipses. 
Made  it  out  to  cut-off  down  toward 
road.  Complete  to  pond  out  by  road. 

Field 

86 

86 

6 

DR 

Complete 

obstHeight=0.75.  hmp  back  and 
forth  w/o  moving,  turned  onto 
obstacle  grade...  need  to 
check! ! ! 

XUV  looked  at  obstacles,  made  one 
pass,  then  turned  around  and 
maneuvered  out  of  cul-de-sac 

Charge  barrels  to  the  north  side.  BU 
a  few  times  and  down  toward  cutoff 
path.  Tur  right  to  a  drop  off  close  to 
road. 

Field 

88 

88 

6 

FCI 

Complete 

complete  until  before  puddle, 
unable  to  find  idisp  capture. 

0.5m  obstacle  height,  XUV  took  one 
pass  by  blockage,  turned  and  took 
high  path  around  safety  truck,  took 
first  rght  turn  (shortcut) 

Two  circles,  took  high  road.  Went 
behind  truck. 

Field 

89 

89 

6 

DR 

Complete 

obst  height  0.75.  e-stopped 
before  cutting  corner  into 
density  grade 

XUV  looked  at  blockage,  probed  a 
little,  HMP  turned  XUV  around, 
maneuvered  out  of  cul-de-sac  via 
shortcut 

Went  to  barrels.  Backup  for  new 
plan.  Back  up  to  north.  Turn  left 
and  gone. 

Field 

107 

106 

6.1 

DR 

Complete 

found 
unblocked 
upper  road 

North  road  open,  stopped  (chase 
could  not  follow)  bot  found 
north  road 

Drove  directly  to  and  up  the  upper 
trail 

Upper  road  open.  Commited  vary 
quickly.  Good. 

Field 

108 

107 

6.1 

FCI 

Complete 

found 
unblocked 
upper  road 

North  road  open,  set  obst  height 
0.5.  stopped( chase  could  not 
follow)  bot  found  north  road 

2:53 

Drove  directly  to  and  up  the  upper 
trail 

Complete  up  the  path. 

Field 

109 

108 

6.1 

AM  only 

DNF 

Circled 

circled,  did  not 
find  unblocked 
upper  road 

North  road  open,  bot  circled  but 
did  not  found  road,  stop  15:08 

Drove  directly  to  barrels  and  bumped 
one,  stopped,  drove  in  circles 
repeatedly,  E.O.M.  declared 

AM  did  not  see  the  upper  path. 

Field 

no 

109 

6.2 

FCI 

Complete 

out  of  culdesac 
but  stuck  later 

egofccellscale=0.1.  The  bot 
went  all  the  way  around  instead 
of  NS  trail.  3:33  stopped  bot 
going  to  pond,  bot  saw  edge  of 
road  as  obstacle 

Drove  to  cul-de-sac,  looked  at 
abarrles,  circled  and  looked  at 
HMMWVs,  took  path  out  of  finger, 
chose  long  route  (because  of  cost  of 
hard  right  turn?) 

Got  out  of  the  finger  after  ony  one 
circle  or  so.  Missed  turn.  Too  tight. 
Around  the  point  it  is  trying  to  get 
back  in.  Ended  run.  0.1  instead  of 

0.2  weight.  Upper  road  blocked. 

Field 

111 

no 

6.2 

FCI 

DNF 

Estop 

out  of  culdesac 
but  stuck  later 

egofcCellScale=0.1.  bot  tried  to 
take  the  closest  NS  trail  but 
turned  around  and  started 
heading  to  a  puddle.  15:52- 
stopped  due  to  puddle 

Circled  in  front  of  obstacles  a  few 
times,  looked  over  bank,  stopped, 
bacup  10m,  took  path  out  of  cul-de- 
sac,  took  first  right  turn  (short  cut), 
turned  around,  came  back  out  of 
short  cut.  E-stop  (headed  for  ditch) 

Circled  around  and  was  headed  out. 
Took  first  path  to  right  and  got  into 
water  trouble.  Weight  was  0.1  not 
0.2.  Upper  road  blocked. 

Field 

112 

111 

6.1 

DR 

Complete 

found 
unblocked 
upper  road 

removing  blockage  from  upper 
trail,  almost  hit  tree.  Stopped 
after  committing  to  the  upper 
trail 

Entered  cul-de-sac,  saw  blockage, 
saw  opening  in  upper  trail  and  took  it 

Came  down,  detected  blockage. 

Path  to  upper  finger  accepted.  Shot 
up  to  it.  Good  film! 

Field 

113 

112 

6.1 

FCI 

Complete 

found 
unblocked 
upper  road 

obst  height  0.5.  good  run. 

Entered  cul-de-sac,  paused  65m  from 
blockage,  was  opening  in  upper  trail 
and  took  it 

PCI  came  to  middle  of  finger. 
Quickly  recognized  blockage  and 
hole  to  upper  trail.  Traveled  to 
upper  trail. 

Field 

114 

113 

6.1 

AM  only 

DNF 

Circled 

circled,  did  not 
find  unblocked 
upper  road 

obst  height  0.75.  circling  around 

XUV  probed  the  blockage,  circled, 
repeated  this  behavior  three  times, 
E.O.M. 

Max  circle  rule  incomplete. 

Field 

115 

114 

1.1 

DR 

DNF 

Estop 

No  NBC, 
backed  into 
weeds  on  rt, 
HMP  problem 

removing  NBC  1 1 :43  stuck 
backing  up 

Barrel  fell  over,  E-Stop,  reposition 
barrel.  Resume,  dove  right,  trying 
turn  around,  backed  into  opening. 
Estop,  Resume,  backup,  HMP 
backing  into  unknown  terrain, 

E.O.M. 

HMP  failure.  No  NBC 

Field 

116 

115 

1.1 

DR 

Complete 

No  NBC, 
found  trail  to 
right,  not  main 
road 

repeat  from  114.  hmp  not 
moving  restarting,  plan 
complete 

Stopped,  backup  (early),  backup, 
took  alternative  path  to  right,  took 
first  left,  dove  left  into  brush,  HMP 
stuck  mode,  killed  HMP,  backup, 
turned  right,  maneuvered  down 
alternative  path,  found  goal 

Complete 

List  of  Symbols,  Abbreviations,  and  Acronyms 


AM 

Autonomous  Mobility 

ARL 

U.S.  Army  Research  Laboratory 

C2 

Command  and  Control 

DR 

Dynamic  Replarmer 

e-stop 

emergency  stop 

FCI 

Field  Cost  Interface 

FTIG 

Fort  Indiantown  Gap 

HMP 

High-Maneuverability  Plarmer 

NBC 

Nuclear,  Biological,  Chemical 

RCTA 

Robotics  Collaborative  Technology  Alliance 

RIVET 

robotic  interactive  visualization  and  exploitation  technology 

TRL 

technology  readiness  level 

TVMA-B 

Tactical  Vehicle  Maneuver  Area-Bravo 

UGV 

unmanned  ground  vehicle 

XUV 

experimental  unmarmed  vehicle 
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COPIES  ORGANIZATION 


I  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
only)  DTIC  OCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BEL  VOIR  VA  22060-6218 

I  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  HRR 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1 197 

I  DIRECTOR 

US  ARMY  RESEARCH  LAB 
RDRL  CIM  L 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1 197 

I  DIRECTOR 

US  ARMY  RESEARCH  LAB 
RDRL  CIM  P 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1 197 

I  DIRECTOR 

US  ARMY  RESEARCH  LAB 
RDRLD 

2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1 197 


ABERDEEN  PROVING  GROUND 

I  DIR  USARL 

RDRL  CIM  G  (BLDG  4600) 
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I  TDCDE  US  ARMY  ARMOR  CTR 

W  MEINSHAUSEN 
BLDG  2200 

BRANDENBURG  STN  RD 
FORT  KNOX  KY  4012 1 -4 1 92 

1  CDR  ARMY  RSCH  OFC 
4300  S  MIAMI  BLVD 
RESEARCH  TRIANGLE  PARK 
NC  27709 

2  CDR  HQ  US  ARMY  TRADOC 
SCI  &  TECHLGY  DIV 
ATFC 

J  KINCAID 
A  CRANE 

FT  MONROE  VA  23651-5850 

2  ARCIC 

MANEUVER  AVN  AND 

SOLDIER  DIVISION 

C  JENKINS 

ETC  S  HATFIELD 

30  INGALLS  RD 

FORT  MONROE  VA  23651 

I  J  STONE 
ATZB-CIC 

CHF  OF  CONCEPTS  DEV 
DIVISION 

FT  BENNING  GA  31905 

4  US  ARMY  TARDEC 
D  THOMAS 
P  DECKER 
J  ERNATT 
D  KOWACHEK 
MS  264 

DAMI  BLDG  200C 
6501  E  ELEVEN  MILE  RD 
WARREN  MI  48397-5000 

I  RS  JPO 

J  JACZKOWSKI 
MS  266 

DAMI  BLDG  200C 
ROOM  1040 

6501  E  ELEVEN  MILE  RD 
WARREN  MI  48397-5000 

I  USASOC  HQ  AOFD  CDT 
S  FORMAN 

2929  DESERT  STORM  DR 
FORT  BRAGG  NC  28310 


NO.  OF 

COPIES  ORGANIZATION 

I  AVN  APPLIED  TECH  DIR 
AMSRD  AMR  AA  I 
K  ARTHUR 
BLDG  401 
LEE  BLVD 

FORT  EUSTIS  VA  23604-5577 

3  NIST 

E  MESSINA 
H  SCOTT 
THONG 
MS  8230 

BLDG  220  RM  123 
100  BUREAU  DR 
GAITHERSBURG  MD  20899 

1  CARNEGIE  MELLON  UNIV 
ROBOTICS  INST 

A  STENTZ 

5000  FORBES  AVE 

PITTSBURGH  PA  1 52 1 3 

6  GENERAL  DYNAMICS  ROBOTICS  SYS 
K  BONNER 
W  BORGIA 
D  RODGERS 
W  DODSON 
J  GONZALEZ 
RDEAN 
1234  TECH  CT 
WESTMINSTER  MD  21 157 

2  ROBOTIC  RSRCH  EEC 
A  LACAZE 

L  SAPRONOV 
STE  300 

555  QUINCE  ORCHARD  RD 
GAITHERSBURG  MD  20878 

I  DIR  USARL 

NASA  GLENN  RSRCH  CTR 
AMSRD  ARE  VT  ET 
M  VALCO 
MS  501-2 

21000  BROOKPARK  RD 
CLEVELAND  OH  44135-3127 
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2  CMDR 

US  ARMY  DTC 
TEDT  TMA 
C  TURNER 
TEDT  TMA 
B  GERMAN 

314  LONGS  CORNER  RD 
APG  MD  21005-5055 

33  DIRUSARL 
RDRL  VT 
M  NIXON 
RDRL  ROI N 
PIYER 
RDRL  SES  E 
RRAO 
RDRL  CII 
B  BROOME 
RDRL  CII  C 
J  DUMER 
T  HANRATTY 
M  THOMAS 
B  BODT  (6  CPS) 

RDRL  CII  A 
S YOUNG 
RDRL  HRS  E 
K  COSENZO 
SHILL 
T  KELLEY 
RDRL  SEE  M 
J  NEALON 
RDRL  VTA 
J  BORNSTEIN 
HEDGE 
M  FIELDS 
C  KRONINGER 
GHAAS 
J  PUSEY 
R  BREWER 
M  CHILDERS  (8  CPS) 
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